Files
nexus/sreweekly/articles/238/03-team-play-with-a-powerful-and-independent-agent-a-full-mission-simulat.html
2026-09-12 17:23:01 +08:00

342 lines
21 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<title>
Team Play with a Powerful and Independent Agent: A Full-Mission Simulation Study
</title>
<meta name="HandheldFriendly" content="True" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<link rel="stylesheet" href="https://resilienceroundup.com/assets/styles.css?v=vRS8ypzsZGjIAbLU">
<link rel="icon" href="https://storage.ghost.io/c/69/23/6923ad11-b2c9-4150-8a9f-9f7a8ea2bf13/content/images/size/w256h256/2023/06/Logo-transparent-1.png" type="image/png">
<link rel="canonical" href="https://resilienceroundup.com/issues/team-play-with-a-powerful-and-independent-agent-a-full-mission-simulation-study/">
<meta name="referrer" content="no-referrer-when-downgrade">
<meta property="og:site_name" content="Resilience Roundup">
<meta property="og:type" content="article">
<meta property="og:title" content="Team Play with a Powerful and Independent Agent: A Full-Mission Simulation Study">
<meta property="og:description" content="Make sure to checkout the link at the bottom for Chaos Conf next week!
Team Play with a Powerful and Independent Agent: A Full-Mission Simulation Study
This week we have a continuation of the series we’ve been followingfrom Nadine Sarter and David Woods. In this issue we’ll">
<meta property="og:url" content="https://resilienceroundup.com/issues/team-play-with-a-powerful-and-independent-agent-a-full-mission-simulation-study/">
<meta property="article:published_time" content="2021-04-04T12:57:53.000Z">
<meta property="article:modified_time" content="2021-04-11T08:09:20.000Z">
<meta property="article:tag" content="Issues">
<meta property="article:publisher" content="https://www.facebook.com/ghost">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="Team Play with a Powerful and Independent Agent: A Full-Mission Simulation Study">
<meta name="twitter:description" content="Make sure to checkout the link at the bottom for Chaos Conf next week!
Team Play with a Powerful and Independent Agent: A Full-Mission Simulation Study
This week we have a continuation of the series we’ve been followingfrom Nadine Sarter and David Woods. In this issue we’ll">
<meta name="twitter:url" content="https://resilienceroundup.com/issues/team-play-with-a-powerful-and-independent-agent-a-full-mission-simulation-study/">
<meta name="twitter:label1" content="Written by">
<meta name="twitter:data1" content="Thai Wood">
<meta name="twitter:label2" content="Filed under">
<meta name="twitter:data2" content="Issues">
<meta name="twitter:site" content="@ghost">
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "Article",
"publisher": {
"@type": "Organization",
"name": "Resilience Roundup",
"url": "https://resilienceroundup.com/",
"logo": {
"@type": "ImageObject",
"url": "https://storage.ghost.io/c/69/23/6923ad11-b2c9-4150-8a9f-9f7a8ea2bf13/content/images/2023/06/Logo-transparent.png",
"width": 60,
"height": 60
}
},
"author": {
"@type": "Person",
"name": "Thai Wood",
"image": {
"@type": "ImageObject",
"url": "https://www.gravatar.com/avatar/259e87655ea449b3f29798ed9b7f197c?s=250&r=x&d=mp",
"width": 250,
"height": 250
},
"url": "https://resilienceroundup.com/author/thai/",
"sameAs": []
},
"headline": "Team Play with a Powerful and Independent Agent: A Full-Mission Simulation Study",
"url": "https://resilienceroundup.com/issues/team-play-with-a-powerful-and-independent-agent-a-full-mission-simulation-study/",
"datePublished": "2021-04-04T12:57:53.000Z",
"dateModified": "2021-04-11T08:09:20.000Z",
"keywords": "Issues",
"description": "Make sure to checkout the link at the bottom for Chaos Conf next week!\n\n\n\nTeam Play with a Powerful and Independent Agent: A Full-Mission Simulation Study\n\n\nThis week we have a continuation of the series we’ve been followingfrom Nadine Sarter and David Woods. In this issue we’ll be examining how automation can create new problems, especially if the operator is expected to keep track of multiple modes with little support.\n\n\nWhile this paper focuses on the A320 aircraft specifically, the authors a",
"mainEntityOfPage": "https://resilienceroundup.com/issues/team-play-with-a-powerful-and-independent-agent-a-full-mission-simulation-study/"
}
</script>
<meta name="generator" content="Ghost 6.64">
<link rel="alternate" type="application/rss+xml" title="Resilience Roundup" href="https://resilienceroundup.com/issues/rss/">
<script defer src="https://cdn.jsdelivr.net/ghost/portal@~2.71/umd/portal.min.js" data-i18n="true" data-ghost="https://resilienceroundup.com/" data-key="85a1e000418a01abe385380c87" data-api="https://resilienceroundup.ghost.io/ghost/api/content/" data-locale="en" crossorigin="anonymous"></script><style id="gh-members-styles">.gh-post-upgrade-cta-content,
.gh-post-upgrade-cta {
display: flex;
flex-direction: column;
align-items: center;
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Open Sans', 'Helvetica Neue', sans-serif;
text-align: center;
width: 100%;
color: #ffffff;
font-size: 16px;
}
.gh-post-upgrade-cta-content {
border-radius: 8px;
padding: 40px 4vw;
}
.gh-post-upgrade-cta h2 {
color: #ffffff;
font-size: 28px;
letter-spacing: -0.2px;
margin: 0;
padding: 0;
}
.gh-post-upgrade-cta p {
margin: 20px 0 0;
padding: 0;
}
.gh-post-upgrade-cta small {
font-size: 16px;
letter-spacing: -0.2px;
}
.gh-post-upgrade-cta a {
color: #ffffff;
cursor: pointer;
font-weight: 500;
box-shadow: none;
text-decoration: underline;
}
.gh-post-upgrade-cta a:hover {
color: #ffffff;
opacity: 0.8;
box-shadow: none;
text-decoration: underline;
}
.gh-post-upgrade-cta a.gh-btn {
display: block;
background: #ffffff;
text-decoration: none;
margin: 28px 0 0;
padding: 8px 18px;
border-radius: 4px;
font-size: 16px;
font-weight: 600;
}
.gh-post-upgrade-cta a.gh-btn:hover {
opacity: 0.92;
}</style><script async src="https://js.stripe.com/v3/"></script>
<script defer src="https://cdn.jsdelivr.net/ghost/sodo-search@~1.8/umd/sodo-search.min.js" data-key="85a1e000418a01abe385380c87" data-styles="https://cdn.jsdelivr.net/ghost/sodo-search@~1.8/umd/main.css" data-sodo-search="https://resilienceroundup.ghost.io/" data-locale="en" crossorigin="anonymous"></script>
<link href="https://resilienceroundup.com/webmentions/receive/" rel="webmention">
<script defer src="/public/cards.min.js?v=ShRHxgy4po8zN-Wf"></script>
<link rel="stylesheet" type="text/css" href="/public/cards.min.css?v=WwnU9jw5ancNC8Gc">
<script defer src="/public/member-attribution.min.js?v=AKG4hWena9j3yX3I"></script>
<script defer src="/public/ghost-stats.min.js?v=vFcCUf6ZQ0Hyhc8h" data-stringify-payload="false" data-datasource="analytics_events" data-storage="localStorage" data-host="https://resilienceroundup.com/.ghost/analytics/api/v1/page_hit" tb_site_uuid="6923ad11-b2c9-4150-8a9f-9f7a8ea2bf13" tb_post_uuid="f7c0ea15-3b25-482f-b656-04dcbefc10fa" tb_post_type="post" tb_member_uuid="undefined" tb_member_status="undefined" tb_gift_link=""></script><style>:root {--ghost-accent-color: #abaaba;}</style>
</head>
<body class="post-template tag-issues tag-hash-wordpress tag-hash-import-2023-02-07-21-23">
<nav class="page-head container">
<div class="head__logo">
<a class="b0 font-heading fsmedium" href="https://resilienceroundup.com">
<img src="https://storage.ghost.io/c/69/23/6923ad11-b2c9-4150-8a9f-9f7a8ea2bf13/content/images/2023/06/Logo-transparent.png" alt="Resilience Roundup" class="logo">
</a>
</div>
<div class="head__nav">
<ul class="nav">
<li class="nav-home"><a href="https://resilienceroundup.com/">Home</a></li>
<li class="nav-consulting"><a href="https://resilienceroundup.com/consulting/">Consulting</a></li>
<li class="nav-issues nav-current-parent"><a href="https://resilienceroundup.com/issues/">Issues</a></li>
<li class="nav-elsewhere"><a href="https://resilienceroundup.com/appearances/">Elsewhere</a></li>
<li class="nav-podcast"><a href="https://resilienceroundup.com/podcast/">Podcast</a></li>
</ul>
</div>
<div class="head__search">
<button class="gh-search-icon" aria-label="search" data-ghost-search style="display: inline-flex; justify-content: center; align-items: center; width: 32px; height: 32px; padding: 0; border: 0; color: inherit; background-color: transparent; cursor: pointer; outline: none;">
<svg width="20" height="20" fill="none" viewBox="0 0 24 24"><path d="M14.949 14.949a1 1 0 0 1 1.414 0l6.344 6.344a1 1 0 0 1-1.414 1.414l-6.344-6.344a1 1 0 0 1 0-1.414Z" fill="currentColor"/><path d="M10 3a7 7 0 1 0 0 14 7 7 0 0 0 0-14Zm-9 7a9 9 0 1 1 18 0 9 9 0 0 1-18 0Z" fill="currentColor"/></svg>
</button>
</div>
</nav>
<div class="container main-content">
<article class="post-full post tag-issues tag-hash-wordpress tag-hash-import-2023-02-07-21-23 no-image no-image">
<div class="post-content">
<h1 class="lhtitle font-heading mb1">Team Play with a Powerful and Independent Agent: A Full-Mission Simulation Study</h1>
<div>
</div>
</div>
<div class="post-content">
<!--kg-card-begin: html--><p>Make sure to checkout the link at the bottom for Chaos Conf next week!</p>
<hr />
<p><a href="https://www.researchgate.net/publication/12195752_Team_Play_with_a_Powerful_and_Independent_Agent_A_Full-Mission_Simulation_Study?ref=resilienceroundup.com">Team Play with a Powerful and Independent Agent: A Full-Mission Simulation Study</a></p>
<p>This week we have a continuation of the series we&rsquo;ve <a href="https://resilienceroundup.com/issues/how-in-the-world-did-we-ever-get-into-that-mode/">been</a> <a href="https://resilienceroundup.com/issues/learning-from-automation-surprises-and-going-sour-accidents/">following</a>from Nadine Sarter and David Woods. In this issue we&rsquo;ll be examining how automation can create new problems, especially if the operator is expected to keep track of multiple modes with little support.</p>
<p>While this paper focuses on the A320 aircraft specifically, the authors are very clear that this is not an issue with this one aircraft or even just aviation in general, but is a problem that can occur when automation doesn&rsquo;t act as a team player.</p>
<p>Automation acts as more of a &ldquo;team player&rdquo; when it acts in a way that allows humans to keep track of what it&rsquo;s doing and anticipate what it will do in the future. The authors even suggest that this measure, how much of a team player a given technology is, can even help predict how effective and successful that it is.</p>
<p>Conversely, when automation isn&rsquo;t a team player, automation can be clumsy and confusing. This particularly true in this case where there is the possibility of &ldquo;mode error,&rdquo; where in one mode some input is ok, but in some other mode unexpected actions occur. This puts a burden on the operator to continue to exercise &ldquo;mode awareness&rdquo; to keep track of what state the automation is in. If this mode awareness is lost, then mode error can occur.</p>
<p>I think this is important for us to keep in mind as we develop tools and systems for others. How often are we expecting users to keep track of what&rsquo;s going on without giving them more than subtle clues or perhaps nothing at all?</p>
<p>And keep in mind, that this study was done with experienced pilots. The authors even setup the study on the simulator so it included some &ldquo;standard proficiency actions&rdquo; which would have helped reveal if it was simply an issue of inexperience. But none of the pilots struggled in that area. I think its common to see some missteps with our technology especially if we&rsquo;re familiar with it or even have built it, to say that many things that aren&rsquo;t outright bugs are &ldquo;training issues&rdquo;. But I think this study and this question of how the automation behaves toward them human team and its ability to integrate can help step out of this mindset.</p>
<p>The 18 experienced pilots were asked to fly 90 minutes on a simulator and were informed about the experiment and had an instructor with them who acted as the captain. Also, pilots were interviewed afterwards and had a chance to ask questions about the experience as well. Aside from the &ldquo;standard&rdquo; bits, the authors explain:</p>
<blockquote>
<p>All other scenario events and tasks were designed to test pilots&#39; awareness of the status and behavior of the automation. The events involved a high potential for surprise arising from a mismatch between actual system behavior and pilots&#39; likely expectations, which are known to guide their system monitoring</p>
</blockquote>
<p>I think there&rsquo;s a temptation to say well, sure, of course if you pick tricky scenarios some pilots will be tricked. But I think that misses the point that the authors didn&rsquo;t change the system, they used a high fidelity A320 simulator. They were able to create these tricky situations because of the way the system exists today, they didn&rsquo;t create the system.</p>
<p>I won&rsquo;t get into all the different ways the flight automation behaved but one really stood out for me. In one of the scenarios the pilots were asked to use the automation after getting clearance for a landing at a runway, &ldquo;24L&rdquo;. Air traffic control gave them clearance and asked them to stay within certain altitude constraints, something which they can program in the flight computer. After doing so, they were informed that they should actually land at runway &ldquo;24R&rdquo;. Doesn&rsquo;t seem like it should be a big change right? But:</p>
<blockquote>
<p>When the pilot changed the runway identifier in the MCDU [Multifunction Control and Display Unit] to 24 R, the automation also erased all previously entered altitude constraints, even though they still applied.</p>
</blockquote>
<p>Not only did it erase that information, it only communicated this erasure by the <em>absence</em> of two icons that were there previously. 4 pilots never noticed this and another 4 were only able to detect this by looking at a different map display. 10 of the pilots were able to notice almost immediately or even anticipate the issue. But even <em>then</em> of the 14, only 12 were able to recover in time by entering the data again. So not only was it hard to detect it was hard to correct!</p>
<p>Another scenario involved a &ldquo;go around,&rdquo; where a pilot would have to abort a landing and try again, but do so at less than 100 feet of the ground after turning off the flight director. This is something that is rare, but especially challenging during a fast paced event. The automation doesn&rsquo;t make this any easier. It turns out this is the only situation where doing the normal procedure <em>does not</em> automatically arm the autothrust system. In this case, pushing the thrust lever as normal to the go around position actually <em>disconnects</em> the autothrust system, making the pilot have to control it manually.</p>
<p>As shocking as this can be to read, especially if we don&rsquo;t have experience in this domain, it gives us a chance to think about our own systems. How often are we doing similar things on our own automation? How can we better make our automation behave as a team player? Often automation is clumsy and gets in the way, and these tense, high tempo situations are precisely where it <em>doesn&rsquo;t</em> help, when the support is needed most.</p>
<h2 id="takeaways">Takeaways</h2>
<ul>
<li>As technologies and automation advances <em>new</em> types of errors can be created, even while other problems are fixed</li>
<li>This means we cannot count on &ldquo;better&rdquo; technology fixing the issues that may exist in a given system.</li>
<li>Without some way for operators to understand the current status of some automation and <em>anticipate</em> it&rsquo;s future state, mode error (where input that would work in one mode causes different behavior in another) becomes likely.</li>
<li>Mode error can occur when inputs are valid in one mode, but cause different behavior in another.</li>
<li>Automation and advancement in technology can create new forms of failure, even while addressing to others.</li>
<li>The more that automation behaves as a team player, the more this sort of error can be avoided.</li>
</ul>
<!--kg-card-end: html-->
<div class="mt4 post-arrows">
<div class="post-nav">
<a href="/issues/building-and-revising-adaptive-capacity-sharing-for-technical-incident-response-a-case-of-resilience-engineering/">&larr; Building and revising adaptive capacity sharing for technical incident response: A case of resilience engineering</a>
</div>
<div class="post-nav tr">
<a href="/issues/the-role-of-error-in-organizing-behaviour/">The role of error in organizing behaviour &rarr;</a>
</div>
</div>
<div class="box mt4 tc">
<h2 style="margin-top:0">Subscribe to Resilience Roundup</h2>
<p>Subscribe to the newsletter.</p>
<form class="subscribe-form mt2" data-members-form="subscribe">
<input type="text" class="subscribe-email" data-members-name placeholder="Your name" autocomplete="false" required>
<input type="email" class="subscribe-email" data-members-email placeholder="your@email.com" autocomplete="false" required>
<button class="button primary" type="submit">
<span class="button-content">Subscribe</span>
<span class="button-loader">
<svg version="1.1" id="loader-1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px" width="30px" height="30px" viewBox="0 0 40 40" enable-background="new 0 0 40 40" xml:space="preserve">
<path opacity="0.2" fill="#000" d="M20.201,5.169c-8.254,0-14.946,6.692-14.946,14.946c0,8.255,6.692,14.946,14.946,14.946
s14.946-6.691,14.946-14.946C35.146,11.861,28.455,5.169,20.201,5.169z M20.201,31.749c-6.425,0-11.634-5.208-11.634-11.634
c0-6.425,5.209-11.634,11.634-11.634c6.425,0,11.633,5.209,11.633,11.634C31.834,26.541,26.626,31.749,20.201,31.749z"></path>
<path fill="#000" d="M26.013,10.047l1.654-2.866c-2.198-1.272-4.743-2.012-7.466-2.012h0v3.312h0
C22.32,8.481,24.301,9.057,26.013,10.047z" transform="rotate(294.22 20 20)">
<animateTransform attributeType="xml" attributeName="transform" type="rotate" from="0 20 20" to="360 20 20" dur="0.5s" repeatCount="indefinite"></animateTransform>
</path>
</svg>
</span>
</button>
<div class="message-success">
<strong>Great!</strong> Check your inbox (or spam folder) for an email containing a magic link.
</div>
<div class="message-error">
Sorry, something went wrong. Please try again.
</div>
</form> </div>
</div>
</article>
</div>
<footer class="container tc fssmall">
<nav class="site-footer-nav">
<ul class="nav">
<li class="nav-sign-up"><a href="#/portal/">Sign up</a></li>
</ul>
</nav>
<p><a href="https://resilienceroundup.com" class="font-heading fsmedium b0">Resilience Roundup</a></p>
<p class="color--grey">&copy; 2026 Thai Wood </p>
</footer>
<script>
let accountLinks = document.getElementsByClassName('nav-account')
Array.from(accountLinks).forEach(link => {
link.style = 'display:none'
})
</script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.5.1/jquery.slim.js" integrity="sha512-1lagjLfnC1I0iqH9plHYIUq3vDMfjhZsLy9elfK89RBcpcRcx4l+kRJBSnHh2Mh6kLxRHoObD1M5UTUbgFy6nA==" crossorigin="anonymous"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/fitvids/1.2.0/jquery.fitvids.min.js" integrity="sha512-/2sZKAsHDmHNoevKR/xsUKe+Bpf692q4tHNQs9VWWz0ujJ9JBM67iFYbIEdfDV9I2BaodgT5MIg/FTUmUv3oyQ==" crossorigin="anonymous"></script>
<script>
/* FitVids */
$(document).ready(function () {
var $postContent = $(".main-content");
$postContent.fitVids();
});
// Gallery layout
var images = document.querySelectorAll('.kg-gallery-image img');
images.forEach(function (image) {
var container = image.closest('.kg-gallery-image');
var width = image.attributes.width.value;
var height = image.attributes.height.value;
var ratio = width / height;
container.style.flex = ratio + ' 1 0%';
})
</script>
<script src="https://unpkg.com/medium-zoom@1.0.6/dist/medium-zoom.min.js"></script>
<script>
mediumZoom(".kg-card img")
</script>
</body>
</html>