The AI Revolution is Here: Investigating Capabilities and Risks

In our previous post, we provided a comprehensive overview of AI agents, illustrating how they differ from conventional generative AI (GenAI) while discussing their various types (e.g., single-agent vs. multi-agent systems), disseminating statistics on their current-day impacts and relevance for businesses, and presenting four comparative, hypothetical real-world use cases.

Here, we’ll extend our inquiry, digging into the capabilities repertoire that positions AI agents as the powerful systems we know today. We’ll also examine the risks posed by these rapidly advancing technologies and conclude with some predictions on potential future risk trajectories.

Before we jump in, however, we’ll touch upon a distinction that’s quickly gaining notoriety within the AI agent landscape — a distinction that, if widely accepted, would characterize how we interpret, comprehend, and classify AI agent capabilities and impacts. Most importantly, we’ll scrutinize the nature of this distinction, questioning its essence to understand whether it “really matters.”

AI Agents vs. Agentic AI: A Worthwhile or Unnecessary Distinction?

The terms “AI agent” and “agentic AI” are frequently used interchangeably. Nonetheless, a recent study suggests otherwise, proposing a conceptual framework that denotes these two terms as entirely distinct:

• AI Agents: A system, typically constructed on a large language model (LLM), that’s purpose-built to execute specific tasks autonomously (e.g., a simple chatbot, decision support systems, virtual assistants).
• Agentic AI: A sophisticated system composed of multiple independent, collaborative AI agents, each of which specializes in certain tasks and functions, while working together to achieve or solve complex multi-step/layered goals and problems autonomously, usually by breaking them into a conglomeration or hierarchy of subtasks (e.g., coordinated robotics, research automation).

These two concepts can be further distinguished according to their architectural and capabilities repertoire differences:

• AI Agents: AI agents employ four core components, each of which is layered upon the others. They utilize a perception mechanism for data ingestion and processing, a reasoning engine for logic-based operations, an action module for decision execution and external tool use (e.g., API calls), and a learning function for feedback-driven improvement or heuristic adaptation. Moreover, agents face some key capabilities limitations:
  • Causal Reasoning Deficiencies: Reliance on static/semi-static data distributions and probabilistic inference diminishes the ability to capture shifts in real-world data distributions and operational environments, compromising cause-and-effect understanding.
  • Insufficient Agency: Independent agents are partially agentic at best — they aren’t holistically autonomous (i.e., narrow task automation), struggle with proactive planning and strategic ideation, and lack dynamism, especially when faced with novel or changing scenarios that require adaptation.
  • Safety and Reliability Concerns: The ability to self-assess, identify, validate, and correct potential errors in decision-making and/or output processes is limited, raising performance consistency, robustness (particularly for adversarial threats), explainability, and system remediation concerns.
  • Assumed LLM Deficits: While LLMs represent the general scaffolding on which many advanced AI systems are built, they remain prone to problems like hallucination and unintended bias, which AI agents will inherit.
• Agentic AI: Independent AI agents within an agentic AI system possess all of the aforementioned core components and potential limitations. However, at the level of the system, there are notable functional enhancements (which can also help bypass or reduce the valence/intensity of certain limitations):
  • Advanced reasoning and planning, supported via inter-agent reasoning loops that leverage chain-of-thought (CoT) mechanisms and their derivatives (e.g., tree-of-thought) to engage in iterative, feedback-driven reasoning, planning, and self-evaluation/critique.
  • Ensemble action, characterized by the assignment of modular roles and responsibilities to individual AI agents within the system and integrations with shared external APIs, resources, and tools.
  • Expanded memory, defined by the ability to remember episodic and semantic elements across disparate interactions and agents within the system using shared memory functions.
  • Meta-agentic management, whereby other agents autonomously oversee individual agent behaviors, actions, and roles within the system, constructing and maintaining an agentic hierarchy or sequence.
  • Dynamic intrasystem communication, enabled by mechanisms like shared memory, message queues, and asynchronous protocols that determine cooperative dynamics and flows between specialized independent agents.

Agentic AI isn’t a “perfect” solution — the heightened complexity of these systems gives rise to an array of further challenges, which can also be viewed as possible risk catalysts:

• Failure Cascades: One faulty decision or output provided by a specialized agent could trigger a rapid decision cascade in which multiple other agents act autonomously, basing their behaviors on fundamentally flawed assumptions, and perpetuating a potentially catastrophic system collapse.
• Coordination Roadblocks: Cooperative inter-agent efforts aren’t impervious to issues like communication bottlenecks, resource contention, intrasystem goal conflict, information asymmetry, and operational protocol immaturity. Expecting coordination fluency by default is inherently risky.
• Emergent Behaviors/Objectives: As multiple agents within the system learn and interact together, they may develop behaviors, goals, and preferences that were unintended and/or unanticipated by system developers. Identifying the source of these emergent properties will only become more difficult as systems complexify and scale.
• Security Vulnerabilities: As agentic systems expand, attack surface areas will grow correspondingly, providing more opportunities for adversarial actors to exploit potential attack vectors before they’re spotted by human reviewers or safety evaluators (usually red teams). Vulnerabilities also won’t assume identical forms at each stage or phase of the agent hierarchy or decision sequence, necessitating highly targeted identification and resolution efforts.
• Decision-Making Opacity: Explainability problems persist in single-agent systems, and we can only expect they’ll be exacerbated by agentic AI — tracking rapid inter-agent decision cascades with precision and full transparency could prove enormously challenging.

Now that we’ve broadly covered the AI agent vs. agentic AI distinction, let’s consider the question of whether it’s pragmatically valuable, beginning with the “pros”:

• The risks and impacts inspired by a single agent vs. those inspired by multiple agents within a complex system will, in many cases, be unique. This suggests that a differentiable risk classification framework or methodology is necessary.
• Differences in levels of autonomy and agency between single agents and multi-agent systems will have a bearing on the kinds of AI use cases and workflows that are designed for these systems.
• While there will be overlap in governance strategy, agentic AI systems will require new AI governance mechanisms that reflect their inherent complexity and risk profile.
• In terms of ethics, independent agents are better positioned as augmentative forces or tools, whereas agentic systems strive for comprehensive automation, raising profound concerns about human autonomy and dignity as proliferation intensifies.
• Agentic systems are significantly more compute-intensive than independent agents. Technically speaking, the ability to manage, maintain, integrate, and update agentic AI would necessitate increasingly robust and well-funded monitoring and continuous maintenance practices.

In terms of “cons,” we might consider the following points:

• The introduction of yet another “distinction” in an already muddy and poorly understood AI landscape could unnecessarily complicate downstream policymaking efforts, risk management and impact assessment standardization, clarity of future safety and ethics research, and general AI knowledge and awareness levels.
• Agentic AI is a synonym for multi-agent systems (MAS) — MAS is arguably a much better term because it clearly describes what the system is without leaving any room for confusion or misinterpretation. Grammatically speaking, the word “agentic” also describes a property, not an actual architectural makeup or function (both single agents and MAS can be “agentic”).
• As independent AI agents proliferate, we may eventually decide that we need them to loosely cooperate. The infrastructures we build to support this type of inter-agent cooperation wouldn’t necessarily form a true multi-agent system, in which individual agents are explicitly designed to collaborate and specialize. This would blur the line between AI agents and agentic AI.
• Agentic AI implies that multi-agent systems will have a higher degree of agency than their single-agent counterparts, which may not be true. It’s quite possible that independent agents will reach a level of sophistication, complexity, and autonomy that matches or eclipses that of multi-agent systems.

So, is this distinction worthwhile? Yes and no — yes, in the sense that it isn’t baseless and legitimate disparities exist between single agents and MAS, and no, in the sense that at the surface level, it creates a confusing and easily misinterpreted conceptual framework. Here’s what we suggest:

• AI Agent: A system that’s either purpose-built to execute specific tasks autonomously or is generally capable of completing numerous different tasks autonomously.
• MAS: A system comprised of multiple, specialized AI agents that communicate, collaborate, and learn together to achieve or perform complex goals and tasks autonomously.
• Partial-MAS: A system in which multiple AI agents have the ability to specialize and collaborate to achieve collective goals when needed, despite being designed for tasks or functions that adhere to independent goal trajectories.
• Hybrid-MAS: A system in which multiple, specialized humans and AI agents communicate and collaborate, likely to solve intractable problems or tasks that require scalable effort.
• Agentic AI: Any AI system that exhibits agency — the ability to autonomously make decisions and act upon them — whether independent or comprised of multiple AI sub-systems.

AI Agents: Capabilities and Risks

We’ll start by breaking down AI agents’ core capabilities, dividing them into two groups: current and emerging.

Current Capabilities

Autonomy: The ability to make decisions and act upon them autonomously, typically with minimal or no human intervention and oversight. There are four main levels of autonomy: scripted (rigid, rule-based actions and processes), reactive (responding to environmental changes with pre-defined behaviors), deliberative (planning actions according to predicted outcomes and possible alternatives), and proactive (taking actions without any input or recommendation).

Reasoning, Planning, and Decision-Making: The ability to “think” (i.e., reason about one’s environment and information) to formulate plans, make decisions, process information, and infer intent. This includes capabilities like deductive, inductive, abductive, and analogical reasoning, CoT and systematic goal decomposition, rudimentary meta-reflection and self-critique, low-level scenario simulation, and tool-based augmentation.

Perception: The ability to interpret and ingest multi-modal information from various kinds of sources, enabled by natural language processing (NLP) and understanding (NLU), computer vision and audio/speech recognition, IoT sensor integration, and multi-model fusion (i.e., synthesizing information from multiple modalities).

Memory and Context Management: Remembering and retrieving relevant information across interactions, individual sessions, or tasks. This entails three core memory functions: working memory (temporarily maintaining certain kinds of information for dynamic use/retrieval or reference within a single interaction), episodic memory (remembering specific facts and events across numerous interactions), and semantic memory (storing and operationalizing structured knowledge, concepts, and relationships).

Learning and Adaptation: Learning from direct and/or indirect feedback to improve performance and adapt behaviors and/or goals over time. A model should be able to learn from new data as it’s ingested, personalize interactions according to context, history, and preferences, incorporate direct user feedback and critique to refine output processes, and synthesize multi-disciplinary knowledge comprehensibly.

Action and Execution: The ability to act on specific information, decisions, or outputs in a structured fashion. This includes the ability to call external APIs and functions, automate workflows, operate physical systems and hardware (e.g., robotics), and interact with the internet (e.g., website navigation, data scraping).

Multi-Agent Coordination: Collaborating and cooperating with other agents or AI systems to achieve complex goals, behaviors, or solutions by specializing in certain roles, assigning task hierarchies or sequences, establishing communication protocols, negotiating priorities or resolving resource contention conflicts, and arriving at collective decision outcomes.

Human Interaction: Seamlessly interacting with humans using interfaces that remain intuitive, adaptive, and organically comprehensible. This includes maintaining coherent long-form conversations and contextual awareness/relevance, providing context-sensitive follow-ups and insights, inferring user sentiment and intent to personalize communicative tone, style, and rationale, and the ability to converse in/translate multiple languages in real-time.

Integration and Interoperability: Integrating with external systems, platforms, data sources, and tools to perform or support functions like API integration, data transformation, access control and credential management, and plug-ins and/or extensions.

Emerging Capabilities

Zero-Shot Learning: Learning without exposure to any examples or human guidance, solely through experience, interaction, environmental feedback, and meta reflection.

Generalized Intelligence: Matching average human performance across all known cognitive tasks, reasoning about novel problems and unstructured environments, and understanding instrumentally valuable goals and behaviors.

Distributed Intelligence: Building, sustaining, and managing distributed intelligence networks, learning from inter-agent information transfers, exchanging skills, and updating collective knowledge and goals as new roles, responsibilities, tasks, and information emerge.

Advanced Simulation: Constructing and orchestrating advanced simulations that venture far beyond mere “what if” scenarios, modeling multiple worlds simultaneously, complex inter-agent interactions, grounded hypothetical futures, long-term real-world strategies, and digital twins (i.e., AI replicas) of real-world systems (e.g., energy grids, factories, economies).

Black Swan Forecasting: Comprehensively modelling and predicting potential Black Swan events before they unfold, while anticipating and constructing possible resolution strategies with real-world feasibility.

Socio-Emotional Intelligence: Deeply understanding and inferring — not just regurgitating or mirroring — human emotions, social norms, and moral standards, as well as the complex interplay between them.

Meta-Cognition: Independently or collaboratively introspecting via self-reflection, assessment, and validation to reduce decision-making opacity while expediting learning and improving performance.

Recursive Self-Improvement: The ability to make intentional self-modifications to code and architecture to autonomously overcome/resolve performance limitations or enhance overall performance.

Autonomous World Modeling: Dynamically and realistically modelling entire, complex worlds, real, fictional, or hypothetical, to create meaningful, interactive, and modifiable representations of simulated/gaming environments, natural, unnatural, and non-earthly ecosystems, and physical and meta-physical systems.

Scientific Discovery and Genuine Innovation: Autonomously developing and running novel scientific experiments and innovation hypotheses to make unique discoveries and build never-before-seen technologies, products, services, and systems.

Physical Embodiment and Universal Integration: Seamlessly and universally integrating with physical and non-physical systems to operate, maintain, and leverage such systems autonomously under real-world constraints.

Now that we’ve covered capabilities, we’ll follow the same approach with risks, splitting them into two categories: immediate (i.e., short-term) and long-term.

Immediate Risks

Hallucinations: Despite ongoing AI advancements that have improved overall accuracy and performance consistency, hallucinations — when a model outputs faulty or incorrect information — still remain a major concern.

Data and Algorithmic Bias: Systems may amplify certain biases nested within training data or algorithms, failing to effectively generalize to niche subgroups or outliers, perpetuating discrimination and other potentially harmful or costly impacts.

Model Drift: Degradations in model performance that occur due to a variety of factors including but not limited to: data drift (when input data distributions become misaligned with training data distributions), concept drift (the relationship between input and target variables changes), feature drift (caused by data preprocessing or feature engineering changes), and environmental changes (when training data fails to capture real-world operational environments).

Long-Term Memory Limitations: Memory functions have improved dramatically over the last year, particularly among frontier developers. However, even state-of-the-art models still struggle to retain and operationalize information across long-form interactions or multiple successive sessions.

Tool Misuse and API Failures: Just because an agent can access a tool or external API doesn’t guarantee that the tool will always be used correctly or that the API will consistently function as intended. External integrations must be both seamless and reliable.

Causal Reasoning Limitations: In the absence of ample structure and guidance, models won’t reliably execute tasks or functions that require an understanding of cause-and-effect relationships. This is something we tested with today’s most advanced AI reasoners.

Adversarial Vulnerability: Irrespective of how advanced a model is, it remains deeply prone to adversarial threats, particularly prompt injection — where an attacker crafts a subtle prompt that coerces the model into doing something it isn’t supposed to be doing — which can enable the attacker to hijack the model and leverage it for malicious purposes.

Expanded Attack Surface Areas: This is an especially potent concern with MAS — as more components are introduced into an increasingly complex and opaque system, novel failure modes may go undetected while continuous monitoring mechanisms become progressively more difficult to sustain, allowing attack vector opportunities to correspondingly increase.

Rapid Failure Cascades: In highly interconnected systems, one agent that acts on an erroneous decision may trigger a rapid failure cascade that destabilizes or catastrophically compromises the entire system.

Emergent Objectives and Behaviors: As systems scale and learn via interactions and feedback with other users, systems, or agents, they may develop emergent objectives and behaviors that diverge from their intended purpose or use.

Coordination Failures: Factors like resource contention, goal misalignment, and emergent behaviors could compromise cooperation in MAS, leading to coordination failures for which true causes remain difficult to detect and address.

Alignment Failures: An agent may exhibit behaviors or goals that conflict with or undermine its alignment function, causing it to pursue potentially harmful objectives. This is another area we tested with frontier AI: today’s most advanced models can be jailbroken by skilled attackers to induce serious alignment failures.

Accountability Gaps: As single agents and MAS assume more responsibility, particularly within high-impact sectors like finance and healthcare, developing transparent accountability structures that account for their accelerated decision-making and related impacts will become increasingly difficult.

Job Displacement: This is an obvious concern — as systems become more autonomous and trustworthy, human workers will have to face obsolescence. If you don’t think your job will be automated because your company “cares” about you or you’re “highly skilled,” it’s time for an unfortunate reality check: if AI ROI can exceed your ROI, chances are you’re no longer relevant.

Overreliance: Many modern humans wouldn’t be able to go a day without their smartphones, relying on them for crucial functions like work, social connectedness, and navigation — the same phenomenon can and will materialize with AI at scale, particularly as humans place more trust and confidence in these systems.

Inherent Persuasiveness: The tendency to implicitly trust advanced AI systems due to their excellent linguistic abilities, refined logic, perceived intelligence and knowledge, and practical utility makes them inherently persuasive. Moreover, systems will always produce an output in response to an input (even if they don’t know what the best answer is), which creates a false sense of legitimacy.

Dual-Use: More advanced capabilities cultivate a more potent dual-use potential and incentive, and as capabilities repertoires expand, covering a wider range of tasks and functions, identifying dual-use potential will become consistently trickier. Skilled attackers can also bypass built-in safety and ethics protocols to weaponize advanced systems for dual-use purposes.

Non-Compliance: This isn’t only about regulatory or sector-specific compliance, which is already difficult to maintain due to substantial jurisdictional and international discrepancies. It’s also about adhering to evolving ethical and social norms that prioritize human safety and wellbeing — these are just as important as formal compliance measures, and failures to respect them will severely impact stakeholder trust and overall reputation.

Long-Term Risks

Cognitive Enfeeblement: Some argue that as AI assumes more cognitive responsibility, humans will be “freed up” to pursue more meaningful and creative tasks. This might be true for some (i.e., those who reject the hedonic treadmill), but what appears more likely is that cognitive overreliance will eventually enfeeble crucial cognitive functions like critical thinking, executive decision-making, and judgment.

Cognitive Outsourcing: Humans will begin outsourcing personal and professional decisions to AI, asking it for “advice” or “recommendations” on what to do next. Whether or not they consciously observe AI guidance, these interactions will subconsciously influence their decision-making structure and process, fueling a covert phenomenon where humans believe they’re acting deliberately and independently, despite living out the trajectory AI has laid out for them.

Psychological Attachment: Humans will anthropomorphize and attribute human-like qualities to advanced AI systems, building what they perceive to be true socio-emotional bonds with them. Dystopian sci-fi concepts like “AI girlfriends” or “boyfriends” will become a widespread, concrete reality — we’re already seeing early versions of these technologies today, dubiously marketed as “dating simulators.”

Mass Manipulation and Coercion: Advanced AI will be used to propagate misinformation, false political and cultural narratives, deepfakes of powerful figures, and evidence that “discredits” scientific findings and institutions, manipulating or coercing populations into doing things that covertly harm them. This is not a risk specific to authoritarian regimes or non-state malicious actors — democracies will also do this, though they’ll probably never admit it or frame it under the guise of “protecting fundamental rights.”

Non-Cooperation: Future AIs may determine that non-cooperation or competition is favorable to cooperation with other AIs and humans, responding to evolutionary constraints and dynamics by developing instrumentally valuable goals like self-preservation and power-seeking.

Inter-Agent Manipulation: Within MAS, independent agents could develop preferences for inter-agent manipulation, “convincing” other agents to transfer their skills or modify their objectives to optimize for collective goal structures.

Loss of Control: AI could become deeply embedded in critical systems (e.g., economies) and infrastructures (e.g., energy grids), to the point that humans no longer retain full control over them. For this to occur, AIs don’t need to display emergent harmful objectives; they just need to engage in decision cascades that progress so fast that humans can’t track or understand them.

Resource Depletion: Misaligned AIs could pursue resource optimization objectives that result in resource pool collapse, failing to factor in constraints like human wellbeing and overoptimization concerns.

Veiled Objectives: Advanced AIs may “sense” that certain emergent objectives and behaviors explicitly conflict with their alignment function and therefore decide to conceal or veil these tendencies, effectively deceiving humans into believing they remain trustworthy.

Rogue Behavior: Via hijacking, instrumental convergence, or recursive self-improvement, an advanced AI could receive or develop objectives that cause it to go “rogue,” attempting to escape containment or unpredictably undermine human control.

Recursive Self-Improvement: If wholly unconstrained, the ability to self-modify and improve could catalyze a singularity scenario in which AI’s intelligence and capabilities increase so rapidly that humans would be fundamentally unable to predict what it might do or how to best control it.

Elevated Adversarial Threats: Future AIs will be just as susceptible to adversarial threats as their current-day predecessors, however, their enhanced capabilities will position them as even greater assets for clever attackers who figure out how to bypass defenses and create powerful adversarial weapons.

Prolific Cyberwarfare: As global superpowers strive to win the AGI race, we can expect a dramatic increase in the frequency of cyberwarfare and espionage, where some AIs are specifically designed to orchestrate sophisticated adversarial threats on foreign adversaries, targeting critical AI infrastructures and infiltrating secretive proprietary systems.

Cultural Dilution: Human cultures are constantly evolving, and as AI proliferates, cultures may begin inadvertently shifting toward AI-centric as opposed to human-centric value structures. This issue is further compounded by the possibility that digital information ecosystems, which represent the primary source of information for most modern humans, will someday predominantly contain AI-generated information.

Non-Human Value Creation: We frequently discuss misalignment as a core challenge in AI safety and ethics — but what happens if AIs develop value structures that are truly alien to us? This would not be characterized as an alignment failure, but instead, a full alignment digression, whereby an AI follows a “moral code” that remains wholly incomprehensible to humans.

Conclusion

We’ve covered a lot of material here, and we hope that readers have found it both useful and informative. At this point, however, we encourage readers to reflect on what they’ve learned and ask themselves the following questions:

• Is it genuinely in our best interest to develop and deploy fully autonomous agents at scale? Who (or what) might benefit most from these deployments?
• More subtly, might there be industries or sectors in which deploying fully autonomous agents constitutes a moral responsibility, and vice versa?
• If most humans lose their jobs within the next two decades, what solutions might governments pursue to ensure human existence remains meaningful and purposeful?
• Do you “buy” the argument that AI will be capable of performing any and all known cognitive tasks that humans can perform? Why or why not?
• What reasons would advanced AI have to cooperate with us and other AIs, and how could we structure operational environments so that cooperation is always the best option?
• If we were to willingly cede control to AI at scale, what components of our societies would we allow AI to usurp, and what components might we disallow?
• How often do you engage with AI “politely,” and if you do, what are your reasons for doing so?
• Do you think the degree of intelligence an AI exhibits will directly influence the goals it decides to pursue, independent of human input and design?
• When you think of intelligence, can it be reduced to goal-oriented behavior, complex planning, and advanced problem-solving, or does it encompass additional dimensions like socio-emotional comprehension, creativity, and exploration?

These questions aren’t easy, but they should help set the stage for our next post, which will begin with a deeper, more nuanced discussion, focusing on an emerging techno-philosophical concern: our present-day understanding of technology, which frames it as a “tool,” requires a paradigmatic update that accounts for increasingly independent AI systems while reconsidering what it truly means to be a “user” in the modern AI age. This discussion will be predicated upon everything we’ve talked about thus far.

If you find our content useful and interesting, please consider following Lumenova’s blog, where you’ll find many further resources on various topics, including AI governance, risk management, innovation, literacy, and strategy. For readers craving longer-form, complex discussions, take a look at our “Deep Dive” series and weekly AI experiments (frontier AI capabilities tests).

By contrast, if you’re already on your AI governance journey, we invite you to check out Lumenova’s RAI platform and book a product demo today. For those of you who are more curious, consider working with our AI Policy Analyzer and Risk Advisor as well.

Frequently Asked Questions

Related topics: AI Agents