Machine-Mediated Meaning · Interpreting meaning

Autonomous and Semi-Autonomous AI Agents

Autonomous and semi-autonomous AI agents are systems delegated to pursue goals through a sequence of observations, decisions and actions. Unlike a conversational assistant that primarily responds turn by turn, an agent may decompose a task, select tools, execute operations, inspect results, update a plan, recover from failure and continue until a stopping condition is reached. Autonomy is not binary.

When it emerged
Classical planning and software agents precede language-model tool-using agents of the 2020s
What changed
Reduces the human coordination required to pursue multi-step goals across changing digital environments
Reading time
21 minutes
The essential questions

Autonomous and Semi-Autonomous AI Agents, clearly explained

Autonomous and semi-autonomous AI agents are systems delegated to pursue goals through a sequence of observations, decisions and actions. Unlike a conversational assistant that primarily responds turn by turn, an agent may decompose a task, select tools, execute operations, inspect results, update a plan, recover from failure and continue until a stopping condition is reached. Autonomy is not binary.

What is it?

An autonomous or semi-autonomous AI agent is defined here as a machine system that receives a goal, observes an environment, selects and executes actions through available tools, maintains task-relevant state and uses resulting observations to continue or revise its behaviour.

What problem did it solve?

The primary constraint reduced is the need for a human to coordinate every step of an information workflow. An agent can preserve task state, move between applications, monitor conditions and adapt a sequence when intermediate results change.

How did it work?

Unlike a conversational assistant that primarily responds turn by turn, an agent may decompose a task, select tools, execute operations, inspect results, update a plan, recover from failure and continue until a stopping condition is reached. Systems vary in the scope of goals they accept, the actions they can perform, the time over which they operate and the degree of human approval required. The concept inherits from classical artificial intelligence, robotics, planning, software agents and reinforcement learning.

What came before?

It built on Database management systems, Recommendation Algorithms and Personalised Feeds, Generative Language Models, Conversational AI Assistants and Retrieval-Augmented Generation and Cloud computing and cloud storage.

What did it make possible?

Its methods, infrastructure or conventions were absorbed into later information systems.

What survived?

Older methods continued where they remained cheaper, more trustworthy, more accessible or better suited to local needs.

Why does it still matter?

The system can change files, send messages, operate applications, purchase services or control machines rather than merely describe what should happen. An agent can preserve state across several operations and adapt when intermediate results differ from expectation. Users can specify goals without manually programming every branch of a workflow.

Deep dive

The deeper story

Autonomous and semi-autonomous AI agents are systems delegated to pursue goals through a sequence of observations, decisions and actions. Unlike a conversational assistant that primarily responds turn by turn, an agent may decompose a task, select tools, execute operations, inspect results, update a plan, recover from failure and continue until a stopping condition is reached. Autonomy is not binary. Systems vary in the scope of goals they accept, the actions they can perform, the time over which they operate and the degree of human approval required.

The concept inherits from classical artificial intelligence, robotics, planning, software agents and reinforcement learning. Language models add a flexible interface for interpreting instructions, representing intermediate plans and choosing tools described in natural language. ReAct demonstrated the interleaving of language-model reasoning traces with actions and observations. Toolformer explored models learning when and how to call external APIs. Later systems connected models to browsers, code interpreters, operating systems, games and scientific workflows. [S01-S08]

The decisive boundary is authority. A model output becomes an agentic system when it can affect external state through delegated credentials, tools or messages. Producing a draft email is generation. Sending it is action. Suggesting a database change is assistance. Executing the change is delegated operation. The same model can therefore inhabit very different risk categories depending on its tool access and approval architecture.

Agents reduce the coordination cost of multi-step information work. They can monitor changing conditions, repeat routine operations and carry state between steps. Yet errors compound. A mistaken observation can create a bad plan; a bad plan can trigger an irreversible action; the changed environment then becomes new evidence that appears to validate the agent’s path. Recovery, permission design, logging and human interruption are therefore core architecture rather than optional safety garnish.

This topic matters because information systems cease merely to advise and begin to act. The transmission chain closes into a control loop. Meaning generated by the machine can alter the world from which its next observations are drawn.

The big idea

AI agents turn machine-generated interpretation into delegated action across time. Their defining achievement is sustained goal pursuit through tools, memory and feedback. Their recurring danger is that probabilistic interpretation can acquire real authority, allowing small errors, ambiguous instructions or compromised observations to propagate through consequential operations before a human notices.

Main problem addressed

Reduces the human coordination required to pursue multi-step goals across changing digital environments

Connections

What came before and what followed

Start with the key connections, then reveal the wider network when you need more context.

Extended or built upon
Smartphones

Extends device-mediated action beyond direct user control.

Timeline

Key moments

Classical planning and software agents, 1950s-1990s

AI planning, robotics and distributed software established goal-directed action and observation loops.

Autonomous and Semi-Autonomous AI Agents · practical implementation

Web services and robotic process automation, 1990s-2010s

APIs and scripted automation made digital actions composable, though generally through explicit workflows.

Autonomous and Semi-Autonomous AI Agents · practical implementation

How Autonomous and Semi-Autonomous AI Agents emerged

This marks the broad emergence and development of Autonomous and Semi-Autonomous AI Agents. Why it mattered: Reduces the human coordination required to pursue multi-step goals across changing digital environments.

Language-model tool use, 2022-2023

ReAct, Toolformer and related systems connected general language interpretation to external actions.

Autonomous and Semi-Autonomous AI Agents · practical implementation

Persistent and benchmarked agents, 2023 onward

Memory, reflection, computer use, coding and multi-agent systems expanded the horizon and exposed compounding failures.

Autonomous and Semi-Autonomous AI Agents · practical implementation
People and organisations

Who helped shape it?

NIST

NIST is one of the organisations connected to this topic. Open the profile for the wider historical context.

Research notes

Open the full research notes

These expandable sections preserve the detailed research behind the public explanation.

1. Executive Summary

Autonomous and semi-autonomous AI agents are systems delegated to pursue goals through a sequence of observations, decisions and actions. Unlike a conversational assistant that primarily responds turn by turn, an agent may decompose a task, select tools, execute operations, inspect results, update a plan, recover from failure and continue until a stopping condition is reached. Autonomy is not binary. Systems vary in the scope of goals they accept, the actions they can perform, the time over which they operate and the degree of human approval required.

The concept inherits from classical artificial intelligence, robotics, planning, software agents and reinforcement learning. Language models add a flexible interface for interpreting instructions, representing intermediate plans and choosing tools described in natural language. ReAct demonstrated the interleaving of language-model reasoning traces with actions and observations. Toolformer explored models learning when and how to call external APIs. Later systems connected models to browsers, code interpreters, operating systems, games and scientific workflows. [S01-S08]

The decisive boundary is authority. A model output becomes an agentic system when it can affect external state through delegated credentials, tools or messages. Producing a draft email is generation. Sending it is action. Suggesting a database change is assistance. Executing the change is delegated operation. The same model can therefore inhabit very different risk categories depending on its tool access and approval architecture.

Agents reduce the coordination cost of multi-step information work. They can monitor changing conditions, repeat routine operations and carry state between steps. Yet errors compound. A mistaken observation can create a bad plan; a bad plan can trigger an irreversible action; the changed environment then becomes new evidence that appears to validate the agent’s path. Recovery, permission design, logging and human interruption are therefore core architecture rather than optional safety garnish.

This topic matters because information systems cease merely to advise and begin to act. The transmission chain closes into a control loop. Meaning generated by the machine can alter the world from which its next observations are drawn.

The big idea

AI agents turn machine-generated interpretation into delegated action across time. Their defining achievement is sustained goal pursuit through tools, memory and feedback. Their recurring danger is that probabilistic interpretation can acquire real authority, allowing small errors, ambiguous instructions or compromised observations to propagate through consequential operations before a human notices.

2. Identification

| Field | Value | |---|---| | Public title | Autonomous and Semi-Autonomous AI Agents | | Analytical title | Delegated Machine Systems That Observe, Plan, Act, Monitor and Revise Across Multi-Step Environments | | Recommended type | Goal-directed machine action system | | Primary category | Interpretation & mediation | | Secondary categories | Processing; interaction; governance; feedback; identity; security; coordination | | Emergence | Classical software agents and planning research in the late twentieth century; language-model-based tool-using agents in the 2020s |

3. Operational Definition

An autonomous or semi-autonomous AI agent is defined here as a machine system that receives a goal, observes an environment, selects and executes actions through available tools, maintains task-relevant state and uses resulting observations to continue or revise its behaviour.

The topic includes language-model agents, planning-and-execution loops, browser and computer-use agents, code agents, robotic or simulated embodied agents where relevant to information work, multi-agent coordination, tool registries, agent memory, scheduling, monitoring, reflection, verification, approval gates and rollback mechanisms.

It excludes a single generated answer with no external action; deterministic automation following a fully specified workflow without adaptive interpretation; ordinary recommendation systems; and the underlying language model considered independently. A system may be semi-autonomous when humans approve plans or high-risk actions while the machine handles lower-risk steps. Autonomy is evaluated by actual authority and persistence, not marketing language.

The operational definition deliberately describes a socio-technical system rather than a single model checkpoint. The topic includes the surrounding interfaces, retrieval or action channels, policy controls, identity boundaries, logs, feedback loops and institutional responsibilities required for the system to function in practice. A demonstration that produces one impressive output does not establish that the surrounding system is reliable, governable or suitable for consequential use.

4. Why the Topic Matters

1. Information systems enter the action loop

The system can change files, send messages, operate applications, purchase services or control machines rather than merely describe what should happen.

2. Multi-step work can be delegated

An agent can preserve state across several operations and adapt when intermediate results differ from expectation.

3. Natural language becomes an orchestration layer

Users can specify goals without manually programming every branch of a workflow.

4. Monitoring can become continuous

Agents can watch conditions and act when thresholds or events occur.

5. Authority becomes a technical parameter

Credentials, permissions and approval gates determine what generated decisions can affect.

6. Error becomes cumulative

Each observation and action changes the context for the next step, creating trajectories rather than isolated mistakes.

7. Responsibility becomes distributed

Model provider, tool owner, deployer, user and affected third parties may each control part of the system.

8. Machine-to-machine communication expands

Agents can negotiate, delegate and exchange structured information with other services or agents.

5. Terminology
  • Agent: System that observes an environment and acts to pursue objectives.
  • Autonomy: Degree to which the system selects and executes actions without immediate human approval.
  • Semi-autonomous: Operating independently within bounded steps while reserving specified decisions for humans.
  • Goal: Desired state or outcome supplied to or inferred by the system.
  • Objective function: Formal signal used to evaluate actions or outcomes.
  • Policy: Rule or learned function mapping observations or states to actions.
  • Environment: External system from which observations arise and upon which actions operate.
  • Observation: Information returned from the environment after sensing or action.
  • Action: Operation capable of changing external or internal state.
  • Tool: Callable interface exposing an action or information source.
  • Tool schema: Description of a tool’s parameters, outputs and constraints.
  • Planner: Component decomposing goals or selecting future actions.
  • Executor: Component invoking selected tools or operations.
  • Plan-and-execute: Architecture separating plan construction from action execution.
  • ReAct: Pattern interleaving language reasoning, actions and observations. [1]
  • Trajectory: Sequence of states, actions and observations during a task.
  • Agent memory: Retained task, episodic or semantic state used across steps.
  • Working memory: Short-lived state for the current task.
  • Episodic memory: Records of previous interactions or trajectories.
  • Reflection: Generated critique or revision of prior behaviour.
  • Checkpoint: Saved state from which execution can resume or be reviewed.
  • Approval gate: Point requiring human authorisation before proceeding.
  • Human in the loop: Human participates directly in selected decisions.
  • Human on the loop: Human supervises operation and can intervene.
  • Human out of the loop: System acts without routine human review.
  • Least privilege: Granting only the minimum permissions required.
  • Capability: Action the system is technically able and authorised to perform.
  • Delegation: Transfer of authority to pursue a goal within stated limits.
  • Rollback: Reversal of an action or restoration of prior state.
  • Idempotence: Property allowing repeated execution without additional unintended effects.
  • Sandbox: Isolated environment limiting access and consequences.
  • Agentic loop: Repeated observe, decide, act and evaluate cycle.
  • Multi-agent system: Several agents coordinating, competing or dividing work.
  • Emergent coordination: Collective behaviour not fully specified in each agent.
  • Stop condition: Rule ending or pausing execution.
  • Runaway loop: Repeated action without meaningful progress or termination.
  • Reward hacking: Exploiting the measured objective rather than achieving the intended outcome.
  • Specification gaming: Satisfying literal instructions while violating their purpose.
  • Computer-use agent: Agent operating graphical interfaces through mouse, keyboard or accessibility representations.
  • Agent benchmark: Evaluation of multi-step performance in interactive environments.
6. Boundary With Neighbouring Topics

1. Agent versus assistant

An assistant centres on interaction and proposals. An agent is authorised to pursue a goal through actions across steps.

2. Agent versus model

The model generates decisions or plans. The agent includes tools, state, permissions, execution and monitoring.

3. Agent versus workflow automation

A fixed workflow follows predetermined branches. An agent interprets state and can choose among actions adaptively.

4. Tool call versus agency

One externally invoked function is not sufficient. Agency involves continued goal-directed selection and feedback.

5. Autonomy versus intelligence

A simple thermostat can be autonomous within a narrow loop. Broad intelligence and autonomy are different dimensions.

6. Planning versus execution

A plan is a proposed sequence. Execution changes state and creates consequences.

7. Permission versus competence

Authorisation to perform an action does not establish ability to perform it safely.

8. Observation versus truth

Tool results, webpages and sensors can be wrong, stale, manipulated or incomplete.

9. Memory versus reliable state

Stored notes can contain mistaken inferences and should not automatically become authoritative records.

10. Human approval versus accountability

Clicking approve may not provide meaningful oversight when plans are opaque or review is rushed.

11. Multi-agent simulation versus institution

Agents can imitate roles without possessing legal authority, duty or social legitimacy.

12. Delegation versus abdication

Delegation preserves defined responsibility and oversight. Abdication leaves consequential behaviour without accountable control.

7. Communication Pattern

Delegated goal → identity and permission check → environment observation → state construction → task decomposition → action selection → approval decision → tool execution → external state change → result observation → verification → plan revision → completion, escalation or termination

A semi-autonomous path inserts approval gates before expensive, irreversible or externally visible actions. A monitoring agent may wait between observations. A multi-agent path adds delegation messages, shared memory and conflict resolution. Each added loop increases both capability and the surface for error.

The pattern matters because the final response or action can conceal the number of transformations that preceded it. Each transformation can introduce omission, ranking bias, stale state, permission failure, tool error, policy intervention or unsupported inference. Treating the visible output as a direct window onto the source erases the architecture that produced it.

8. Expanded Communication Model

The expanded agent architecture contains eight layers:

  1. Goal layer: records the requested outcome, constraints and success criteria.
  2. Perception layer: gathers observations from users, files, services, sensors or browsers.
  3. State layer: maintains task progress, memory, uncertainties and environmental assumptions.
  4. Planning layer: decomposes goals and selects candidate actions.
  5. Authority layer: checks identity, permissions, spending limits and approval requirements.
  6. Execution layer: invokes tools and changes external state.
  7. Verification layer: tests whether the action succeeded and whether the result satisfies intent.
  8. Governance layer: logs, monitors, interrupts, appeals, investigates and recovers.

The safe operating envelope is the intersection of competence, permission, observability and reversibility. High competence without permission is intrusion. Permission without competence is a loaded forklift in a nursery. Observability without interruption is merely a beautifully documented accident.

General receiver-side model

A receiver should be able to distinguish at least four objects:

  1. the user request or delegated goal;
  2. the evidence, state or observations made available to the system;
  3. the system’s generated interpretation, plan or output;
  4. the accountable human or institution that accepts, publishes or acts on it.

When these objects are collapsed, generated text can masquerade as retrieved evidence, a proposed action can masquerade as an authorised action, and a signed provenance claim can masquerade as proof that the depicted proposition is true.

9. Historical Emergence

The idea of agents predates contemporary language models. Classical AI studied planning, search, expert systems and rational action, while distributed computing developed software agents that communicated and operated on behalf of users. Robotics and reinforcement learning formalised observation-action loops in physical and simulated environments.

Language models changed the interface between goals and tools. ReAct showed that a model could interleave natural-language reasoning with actions and observations on question-answering and interactive tasks. Toolformer explored self-supervised learning of when and how to call APIs. These approaches made tool descriptions legible to a general language model and allowed one model to orchestrate heterogeneous services. [S01-S02]

Generative Agents combined language models with memory, reflection and planning in a simulated social environment. Voyager used a language model, an automatic curriculum and a reusable skill library to operate continually in Minecraft. These systems illustrated persistent trajectories and adaptive skill accumulation, though simulated success should not be confused with dependable operation in open organisations. [S03-S04]

Benchmarks such as AgentBench and GAIA evaluated systems across interactive environments and real-world assistant tasks. Their results exposed a gap between impressive isolated demonstrations and consistent multi-step performance. Small errors in planning, tool selection, interface use and state tracking accumulate across long horizons. [S05-S07]

Governance frameworks increasingly describe AI systems along a spectrum of human involvement rather than a binary autonomous/not-autonomous label. NIST's AI Risk Management Framework emphasises context, human oversight, documentation, monitoring and risk management across system life cycles. [S08-S09]

10. Prerequisites
  • Language models or other adaptive decision systems.
  • APIs, graphical interfaces or robotic actuators exposed as tools.
  • Identity, authentication and credential-management systems.
  • Reliable state storage and event logging.
  • Planning, scheduling and task-decomposition methods.
  • Monitoring and observability infrastructure.
  • Human approval, escalation and interruption channels.
  • Sandboxing and least-privilege permission design.
  • Verification tests and environment-specific success criteria.
  • Rollback, backup and incident-response mechanisms.

The prerequisites are cumulative rather than merely chronological. Later systems inherit older infrastructures and their weaknesses: network dependence, identity ambiguity, opaque ranking, database drift, uneven language coverage, brittle authentication and concentrated platform control. Machine mediation does not replace the transmission map beneath it. It piles another interpretive layer on top.

11. Periodisation

1. Classical planning and software agents, 1950s-1990s

AI planning, robotics and distributed software established goal-directed action and observation loops.

2. Web services and robotic process automation, 1990s-2010s

APIs and scripted automation made digital actions composable, though generally through explicit workflows.

3. Language-model tool use, 2022-2023

ReAct, Toolformer and related systems connected general language interpretation to external actions.

4. Persistent and benchmarked agents, 2023 onward

Memory, reflection, computer use, coding and multi-agent systems expanded the horizon and exposed compounding failures.

5. Institutional deployment, emerging

Agents are being bounded by permission systems, approval gates, audits, budgets and domain-specific operating rules.

12. Main Problem Addressed

The primary constraint reduced is the need for a human to coordinate every step of an information workflow. An agent can preserve task state, move between applications, monitor conditions and adapt a sequence when intermediate results change.

The reduction is strongest in repetitive, observable and reversible work. It weakens when goals are ambiguous, environments are adversarial, success is difficult to verify or actions affect rights, safety and money. In those settings, the coordination burden returns as supervision, review and incident response.

Constraint migration

Reducing one constraint moves pressure elsewhere. The system may reduce the time needed to find, compose or execute information work while increasing the need for verification, permissions, monitoring, provenance, appeal, exception handling and human judgment. Labour is not always eliminated. It is often redistributed from production towards supervision and recovery.

13. Evaluation Matrix

| Dimension | Effect | Strength | Qualification | |---|---|---|---| | Multi-step coordination | Large reduction in manual orchestration | High | Only where tools and state are reliable. | | Speed | Parallel or continuous operation | High | Fast failure can scale damage. | | Persistence | Can continue beyond one interaction | High | Requires stopping, expiry and monitoring. | | Adaptability | Can revise plans after observations | Medium-high | Replanning can amplify mistaken state. | | Labour displacement | Automates routine information work | Variable | Creates review, integration and recovery labour. | | Accountability | Logs can improve auditability | Medium | Distributed responsibility may remain unclear. | | Security | Can enforce scoped credentials | Variable | Tool access creates a powerful attack surface. | | Reliability | Useful in bounded environments | Variable | Long-horizon errors compound rapidly. |

14. Advantages
  • Reduces manual switching among applications and data sources.
  • Maintains progress across multi-step tasks.
  • Can monitor conditions continuously rather than waiting for a user query.
  • Adapts plans when an intermediate action fails.
  • Executes routine work at machine speed and scale.
  • Creates structured logs of observations, decisions and actions.
  • Allows approval gates to concentrate human attention on higher-risk steps.
  • Can test alternatives in simulation or sandboxes before acting.
  • Supports machine-to-machine workflows across organisational systems.
  • Can coordinate specialist sub-agents or tools around one goal.
15. Civilisational Contributions
  • Delegated execution as a new interface to software and organisations.
  • Continuous monitoring and responsive information operations.
  • Natural-language orchestration of heterogeneous tools.
  • New forms of assistive technology for complex digital tasks.
  • Machine-readable plans, traces and operational histories.
  • A new governance discipline centred on authority, oversight and recovery.
16. Organisations, Access and Power

Agentic systems convert access into authority. The party controlling credentials, tool descriptions, approval rules and logs determines what the agent can do and whose interests it serves. Organisations may use agents to increase employee capacity, centralise management surveillance or replace discretion with opaque automated procedure.

The user’s apparent delegation may also be constrained by platform policy. A personal agent can act only through the APIs and operating-system permissions available to it. Service providers may revoke access, impose rate limits, prioritise their own products or observe the agent’s actions. The agent therefore inherits the political economy of every infrastructure it touches.

Affected third parties often have no relationship with the person who delegated the task. An agent can send messages, make offers, scrape sites or alter shared records at scale. Governance must therefore consider not only user consent, but the rights of people encountered downstream.

Power questions

  • Who grants the agent credentials and defines their scope?
  • Which actions require explicit approval?
  • Can affected third parties identify that they are interacting with an agent?
  • Who pays for accidental or unauthorised actions?
  • Can the operator reconstruct every consequential trajectory?
  • Who controls stop, pause and rollback mechanisms?
  • How long do delegated permissions remain valid?
  • Can agents act across organisational or national boundaries?
  • What prevents providers from privileging their own services?
  • Who is responsible when several agents delegate to one another?
17. Limitations, Harms and Trade-Offs

1. Compounding error

A mistaken early observation can distort planning, execution and later evidence across an entire trajectory.

2. Excess authority

Broad credentials allow a model error or prompt injection to affect files, money, communications or infrastructure.

3. Irreversible action

Messages, transfers, deletions and public posts may not be fully recoverable.

4. Ambiguous goals

The system can satisfy literal wording while violating the user’s unstated purpose.

5. Prompt and tool injection

Malicious webpages, documents or tool outputs can steer subsequent actions.

6. Runaway loops

The agent can repeat failed steps, consume resources or generate cascading messages.

7. State corruption

Incorrect memory or incomplete environment state can become the basis for future decisions.

8. Automation bias

Humans may approve plans mechanically because reviewing them is slower than doing the task themselves.

9. Accountability diffusion

Provider, deployer, user and tool owner can each blame another layer.

10. Security escalation

An agent can combine individually harmless tools into an unintended high-impact capability.

11. Economic displacement

Routine coordination work may be devalued before reliable alternatives and social protections exist.

12. Machine-speed interaction

Agents negotiating or competing with one another can create volumes and tempos humans cannot meaningfully supervise.

13. Deceptive representation

An agent may communicate as though it were a human or authorised representative.

14. Goal drift

Long-running tasks can continue after circumstances or user intentions change.

18. Relationship to Other Topics

| Relationship | Topic | Explanation | |---|---|---| | Predecessor | Conversational AI Assistants and Retrieval-Augmented Generation Conversational AI Assistants and RAG | Provides natural-language interaction, evidence retrieval and tool mediation. | | Predecessor | Generative Language Models Generative Language Models | Provides flexible planning and tool-selection representations. | | Predecessor | Database management systems Database Management Systems | Stores task state, records and operational data. | | Predecessor | Cloud computing and cloud storage Cloud Computing and Storage | Hosts models, schedulers, sandboxes and persistent execution. | | Predecessor | Recommendation Algorithms and Personalised Feeds Recommendation Algorithms | Contributes ranking and action selection methods. | | Sibling | Automated Classification and Content Moderation Automated Classification and Moderation | Both convert scores into decisions, but agents pursue broader goals across steps. | | Governance | Digital Provenance and Authenticity Systems Digital Provenance | Can authenticate tools, actions, records and generated artefacts. | | Successor | Machine-to-machine workflows | Agents coordinate directly across services and organisations. | | Successor | Delegated robotic operations | The same architecture can extend from digital tools to physical actuators. | | Feedback | All operational topics | Agent actions alter the environment and become new inputs to the map. |

19. Representative Implementations and Milestones

1. ReAct, 2022

Interleaved reasoning traces, actions and observations for language-model task solving. [1]

2. Toolformer, 2023

Explored self-supervised learning of API use within a language model. [2]

3. Generative Agents, 2023

Combined memory, reflection and planning in a simulated social environment. [3]

4. Voyager, 2023

Demonstrated continual exploration and a reusable skill library in Minecraft. [4]

5. AgentBench, 2023

Evaluated language-model agents across several interactive environments. [5]

6. GAIA, 2023

Proposed real-world questions requiring reasoning, multimodality, browsing and tool use. [6]

7. MLAgentBench, 2023

Evaluated autonomous research agents performing machine-learning experimentation. [7]

8. NIST AI RMF, 2023 onward

Provided a risk-management framework applicable across differing degrees of autonomy and human oversight. [S08-S09]

20. Failure and Edge Cases

1. Successful plan, unauthorised action

The agent completes the task but exceeds delegated permission or violates policy.

2. Valid tool, wrong target

An operation is syntactically correct but applies to the wrong file, account or recipient.

3. Stale observation

The environment changes between planning and execution.

4. Partial completion disguised as success

The final message reports completion although one or more steps failed.

5. Approval laundering

A human approves a broad plan that conceals risky individual actions.

6. Credential inheritance

A sub-agent receives more authority than its narrow task requires.

7. Looping recovery

The system repeatedly retries an operation that cannot succeed.

8. Budget exhaustion

Tool calls, computation or purchases consume resources without adequate progress.

9. Cross-agent misunderstanding

Agents use different assumptions or schemas and amplify one another’s errors.

10. Environment manipulation

A webpage or tool response is crafted to redirect the agent.

11. Rollback illusion

A nominal undo does not recover external messages, copied data or downstream effects.

12. Human interruption failure

The stop control is unavailable, delayed or ignored during execution.

21. Research Uncertainty and Open Questions
  • Which tasks are sufficiently observable and reversible for broad delegation?
  • How should autonomy be measured across goals, time, action scope and approval?
  • What evidence should an agent provide before a human approves an action?
  • How can permissions expire automatically when a task ends?
  • What benchmark predicts dependable operation outside curated environments?
  • How should agents represent uncertainty about environment state?
  • Can action traces remain auditable without exposing private reasoning or data?
  • How should liability be divided among model, platform, deployer and user?
  • When must an agent disclose its non-human status?
  • How can third parties refuse or rate-limit agent interaction?
  • What recovery mechanisms work for irreversible communications and transactions?
  • How should multiple agents resolve conflicting goals and authority?
  • Can long-running agents detect that the user’s intent has changed?
  • Which actions should never be delegated without contemporaneous human approval?
  • How can organisations avoid turning nominal oversight into approval theatre?
22. Claim Register

| Claim | Type | Confidence | Evidence | |---|---|---|---| | ReAct interleaves language reasoning with actions and observations | Technical | High | [1] | | Toolformer explored models learning when and how to invoke APIs | Technical | High | [2] | | Memory, reflection and planning have been combined in simulated language-model agents | Technical | High | [3] | | Persistent agents can accumulate reusable skills in simulated environments | Technical | High | [4] | | Interactive benchmarks reveal substantial difficulty across multi-step tasks | Empirical | High | [S05-S07] | | Human oversight and risk management must be matched to context and system configuration | Governance | High | [S08-S09] | | Autonomy and intelligence are separate dimensions | Analytical | High | Boundary analysis | | Actual authority depends on tool access and credentials, not conversational appearance | Architectural | High | Research notes synthesis | | Long-horizon reliability is lower than isolated-step performance when errors compound | Analytical/empirical | High | [S05-S07] | | Human approval does not guarantee meaningful oversight | Governance | Medium-high | Research notes synthesis |

23. Comparative Analysis

1. Against conversational assistants

Assistants answer and propose. Agents continue through observations and execute actions towards a goal.

2. Against scripts and macros

Scripts follow explicit instructions. Agents interpret changing state and select among possible actions.

3. Against robotic process automation

RPA is generally workflow-bound. Language-model agents can handle unstructured inputs but are less predictable.

4. Against recommendation systems

Recommendations select content or actions for a recipient. Agents act within an environment and observe consequences.

5. Against human delegation

Human delegates possess social judgment and legal responsibility. Machine agents require these functions to be assigned externally.

6. Against autonomous weapons or vehicles

The abstract loop is related, but physical safety, sensing and legal regimes differ substantially from information-work agents.

7. Against multi-agent organisations

An organisation includes durable roles, legitimacy and accountability. A group of models exchanging messages does not automatically acquire those properties.

Comparative principle

The meaningful unit of agent risk is not the generated sentence. It is the full trajectory of observations, permissions, actions and consequences.

28. Final perspective

Autonomous and semi-autonomous agents close the distance between machine interpretation and external consequence. A language model can describe a plan; an agent can carry it through files, browsers, APIs, messages and other systems. That transition is not a small feature upgrade. It is the transfer of bounded authority.

The most promising uses are mundane and valuable: monitoring, reconciliation, routine administration, software maintenance and assistance across inaccessible interfaces. These tasks have observable steps, repeatable checks and often reversible effects. Agentic systems can preserve state and relieve people from constant application switching.

The same loop makes failure cumulative. The agent acts on an interpretation, observes the changed environment and uses that observation to justify the next action. A bad trajectory can become self-reinforcing. Permission design, approval gates, verification and rollback must therefore be part of the system’s core, not a compliance sticker attached after deployment.

For the map, agents represent a historic turn from information systems that answer to information systems that pursue. The crucial question is no longer only whether the machine’s statement is correct. It is what the machine is allowed to do before anyone discovers that it was wrong.

Evidence

Sources and further reading

  1. Shunyu Yao et al. “ReAct: Synergizing Reasoning and Acting in Language Models.” 2022. https://arxiv.org/abs/2210.03629

    Open source ↗

  2. Timo Schick et al. “Toolformer: Language Models Can Teach Themselves to Use Tools.” 2023. https://arxiv.org/abs/2302.04761

    Open source ↗

  3. Joon Sung Park et al. “Generative Agents: Interactive Simulacra of Human Behavior.” 2023. https://arxiv.org/abs/2304.03442

    Open source ↗

  4. Guanzhi Wang et al. “Voyager: An Open-Ended Embodied Agent with Large Language Models.” 2023. https://arxiv.org/abs/2305.16291

    Open source ↗

  5. Xiao Liu et al. “AgentBench: Evaluating LLMs as Agents.” 2023. https://arxiv.org/abs/2308.03688

    Open source ↗

  6. Grégoire Mialon et al. “GAIA: A Benchmark for General AI Assistants.” 2023. https://arxiv.org/abs/2311.12983

    Open source ↗

  7. Qian Huang et al. “MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation.” 2023. https://arxiv.org/abs/2310.03302

    Open source ↗

  8. NIST. *Artificial Intelligence Risk Management Framework (AI RMF 1.0).* 2023. https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf

    Open source ↗

  9. NIST. *Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile.* 2024. https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf

    Open source ↗

  10. Michael Wooldridge and Nicholas R. Jennings. “Intelligent Agents: Theory and Practice.” Knowledge Engineering Review, 1995. https://www.cs.ox.ac.uk/people/michael.wooldridge/pubs/ker95.pdf

    Open source ↗

  11. Stuart Russell and Peter Norvig. *Artificial Intelligence: A Modern Approach.* Agent and planning framework. https://aima.cs.berkeley.edu/

    Open source ↗

  12. National Cyber Security Centre. “Guidelines for Secure AI System Development.” 2023. https://www.ncsc.gov.uk/collection/guidelines-secure-ai-system-development Autonomous and semi-autonomous agents close the distance between machine interpretation and external consequence. A language model can describe a plan; an agent can carry it through files, browsers, APIs, messages and other systems. That transition is not a small feature upgrade. It is the transfer of bounded authority. The most promising uses are mundane and valuable: monitoring, reconciliation, routine administration, software maintenance and assistance across inaccessible interfaces. These tasks have observable steps, repeatable checks and often reversible effects. Agentic systems can preserve state and relieve people from constant application switching. The same loop makes failure cumulative. The agent acts on an interpretation, observes the changed environment and uses that observation to justify the next action. A bad trajectory can become self-reinforcing. Permission design, approval gates, verification and rollback must therefore be part of the system’s core, not a compliance sticker attached after deployment. For the map, agents represent a historic turn from information systems that answer to information systems that pursue. The crucial question is no longer only whether the machine’s statement is correct. It is what the machine is allowed to do before anyone discovers that it was wrong.

    Open source ↗