Agents and multi-agent systems (MAS) have evolved from
autonomous problem-solving entities to integrating generative
AI and LLMs, ultimately leading to cooperative AI frameworks
that enhance adaptability, scalability, and collaboration. 
**Main Takeaways**
* Multi-agent systems have evolved from rule-based autonomy to cooperative AI,
emphasizing collaboration, negotiation, and ethical alignment.
* The rise of Agentic AI, driven by LLMs, introduces new opportunities for flexible
decision-making but raises challenges in efficiency and complexity.
* Integrating cooperative AI with generative models requires balancing
adaptability, transparency, and computational feasibility in multi-agent
environments 

**Context & History** 
The field of multi-agent systems emerged in the late 1980s/early 1990s,
with its main influences coming from two disparate areas [1,2]. One was the
field of AI robotics, which had begun to seriously address the issue of integrated agent architectures: how do we assemble several cognitive components of intelligence (planning, reasoning, learning, vision,…) into an integrated computational agent? The second was the nascent area of distributed AI, which studied how multiple AI systems could be made to solve problems cooperatively, by dynamically sharing information and tasks. By the mid1990s, these ideas had given rise to the new field, driven by the vision of having (semi)autonomous AI systems – agents – working on behalf of individual users in pursuit of their users’ goals, possibly interacting with other such agents in order to do so. A key insight
was that, since delegated goals might not necessarily be in harmony, it would be necessary to equip such agents with the ability to reason socially. Thus, while AI historically emphasised components of intelligence such as reasoning and problem-solving, the new field of multi-agent systems emphasised social skills such as cooperation, coordination, argumentation and negotiation. In order to underpin those skills, the development of models of Theory of Mind became central to the field. 
By the late 1990s, the field had its own conferences and journal, and was
firmly established as a key sub-field of AI. The area flourished from the late
1990s, with enormous energy devoted to (e.g.) communication languages
for autonomous agents, protocols for cooperation, coordination, and
negotiation, and the underpinning theory of these social skills. With
respect to the latter, while in the early years the practical reasoning paradigm of AI planning had been the dominant influence on the theory of multiagent systems, by the early part of this century, game theory had become the dominant theoretical foundation. Game theory, which emerged from the field of economics, is the theory of interaction between self-interested agents. Although originally devised as a tool for studying interactions between humans and human organisations, it nevertheless seemed a natural framework for studying interactions between artificial agents. A huge body of work emerged, studying (for example), how auctions might be used to allocate scarce resources, the theory
of negotiation between self-interested artificial agents, and how agents might optimally form teams to solve problems and share the associated benefits of cooperation. Interestingly, although learning in multi-agent systems was a key component of the field from the outset, it was not the centre of attention within the field in the first decade or so. 
This initial boom period for multiagent systems lasted roughly from
the mid 1990s to around 2010-15. By the end of that time, though,
some uncomfortable questions were beginning to be asked. While the field
had generated impressive quantities of scientific results, applications seemed to be thin on the ground. For sure there were some high-profile applications. The field of security games, which emerged from multi-agent systems, used ideas from game theory to allocate scarce security resources to defend high-profile targets such as airports. This work led to deployed
applications at US airports and ports. Automated high-frequence trading
systems, which plan and execute the bulk of trades on the world’s markets,
are multi-agent systems on a global scale. And agent-based modelling,
which models socio-technical systems at the level of individual decisionmakers, received a huge boost after the 2008 financial crisis, and again after the 2020 COVID-19 pandemic, where it was demonstrated to be an important tool for modelling the spread of contagion: financial in the first case; epidemiological and social policy in the second. But for all these successes, the core vision of multi-agent systems - where agents function in the context of other agents is an active area of research, exploring social concepts such as norms, organisations, practices and also values, is ongoing work, but for a large part outside the AAMAS community, but within social simulation research, and with results informing and
shaping policy-making in several areas from public health to transportation,
and urban transformation. 
While applications of multi-agent systems research (AI agents interacting
with other AI agents) has not, as yet, lived up to early expectations,
individual dialogue agents such as Alexa, Siri, Cortana are now an everyday
reality, and trace their historical roots both to work on intelligent agents
in the 1990s and work from the NLP community on dialog systems. Many
other applications of this thread of work have achieved success over the past three decades: automated call center assistants, customer service assistants, smartphone virtual assistants, smart speaker assistants, home robot assistants, that can converse with human users and accomplish tasks like ticket booking, restaurant reservations, online shopping, medical and health assistance, and sales assistance, empowered by AI modules like
speech recognition, natural language understanding, dialog management (i.e. state tracking and dialog policy), and natural language generation and speech synthesis. 
As ML surged in the early part of this century, activity in this area increased
within the multi-agent systems field. Multi-agent reinforcement
learning (MARL) grew to become the single biggest area within the field,
possibly driven in part by the fact that developing MARL experiments can be done relatively quickly and without recourse to expensive hardware. At the time of writing, while MARL represents a significant sub-field of ML as a whole, it seems to lack any clear unifying vision or direction – or application. 
**Current State & Trends** 
The emergence of LLMs from 2020 onwards has also led to increased
interest in agents [3]. LLMs can be used as part of a workflow to
automate routine tasks, and the general capabilities of such “agents” for
planning and problem solving is widely discussed. In this context, the concept of Agentic AI refers to the integration of generative AI and LLMs into autonomous agent frameworks aiming to leverage the generative capabilities of such models to enhance interaction, creativity, and real-time decision making in dynamic environments. As we write this (late 2024) there has been an explosion of startup companies hoping to commercialise such agents. Despite this renewed enthusiasm, the original aims of AAMAS from 30 years ago, such as building robust, autonomous multiagent systems capable of complex coordination and long-term reasoning, have not been fully realized. The extent to which this new wave of agent activity is informed by what went before is unclear. 
The challenge now is to understand what multi-agent systems mean in
the era of LLMs. The current direction of agentifying LLMs may lead to
overly complex and unnecessary architectures and heavy computational
costs, whereas adopting a multi-agent paradigm to the development and use of LLMs may offer a sustainable way to compose, diversify, and integrate approaches effectively. Even though distribution was one of the original drivers for the MAS field, this is still a largely unexplored direction under the current paradigm. Another trend nowadays is to recover ideas from classical cognitive architectures to add common sense skills to autonomous agents. 
An emerging trend is multi-agent architectures, which structure
AI components into modular systems that improve transparency,
adaptability, and ethical alignment. The focus on cooperative agents
highlights a shift toward AI that prioritizes collaboration, negotiation, and shared decision-making. By applying modularity, encapsulation,
and separation of concerns, these architectures enable scalable teamwork
between autonomous agents and humans, making them ideal for
hybrid AI applications requiring trust, explainability, and domain-specific
expertise. 
**Research Challenges** 
* Identify challenges and benefits of embedding GenAI-driven agents
into MAS, focusing on enhancing collaboration without disrupting
existing dynamics.
* Investigate how LLM-powered agents can improve negotiation and
decision-making in dynamic multiagent environments while ensuring
ethical alignment and safety.
* Develop architectures that integrate LLM-driven agents while maintaining
scalability, transparency, and computational efficiency in multiagent settings. 
**Community Opinion** 
The survey responses indicate a
majority of respondents finding this
theme relevant to their research, with
a growing interest in integrating Large
Language Models (LLMs) into multiagent systems. Many participants
already use AI agents, with LLMs being
the most common technique (29.34%),
highlighting their expanding role in AI driven applications. 
The potential of multi-agent systems
leveraging LLMs is seen in areas such
as collaborative problem-solving
(68.86%), distributed decision-making
(54.49%), and social simulations
(41.32%). However, challenges persist,
including misalignment between LLMs’
general knowledge and specific system
needs (59.88%), lack of interpretability
(59.28%), and security risks (50.90%).
These concerns suggest a need for
improved explainability, alignment
strategies, and robust security measures
to ensure effective deployment. 
There is also a debate on the necessity
of agentifying LLMs—while 51.5%
believe multi-agent LLM paradigms
are essential for sustainable AI,
42.33% disagree that they introduce
unnecessary complexity. The
computational cost-benefit balance
remains uncertain, with responses
divided on whether LLMs outweigh
their costs. 
The textual responses highlight a broad
spectrum of perspectives on integrating
Large Language Models (LLMs) into
multi-agent systems (MAS), with some
advocating for hybrid approaches
rather than relying solely on LLMs. Many
respondents stressed the need for
diverse AI architectures, emphasizing
modular, multi-technology systems
where LLMs play a role but do not
dominate. Governance, coordination,
and adaptability emerge as key
advantages of MAS, while concerns
include increased complexity, lack
of theoretical guarantees, and high
computational costs. Several responses
criticize the overemphasis on LLMs,
questioning whether they are truly
essential or merely a current trend.
Others highlight practical challenges
such as grounding, alignment, and
robust communication protocols,
pointing out the need for new
frameworks that integrate symbolic
reasoning, structured governance,
and scalable architectures. Overall,
the discussion reflects a critical but
open stance toward agentifying LLMs,
suggesting that context, application
domain, and technological diversity will
shape their effectiveness in multi-agent
environments. 
In summary, the survey reflects
optimism about LLM-driven multiagent systems, but also underscores
the need for addressing key challenges
before widespread adoption.
 
1. Yoav Shoham. Agent-oriented programming. Artificial Intelligence. Artificial Intelligence. 60 (1): 51–92.
2. Michael Wooldridge. An Introduction to Multi-agent Systems 2e. Wiley, 2009.
3. Julia Wiesinger, Patrick Marlow and Vladimir Vuskovic. Agents. Google whitepaper. https://archive.org/details/google-ai-agents-whitepaper

AI agents

The ability to reason has been a salient characteristic of human
intelligence, and there is a critical need for verifiable reasoning
in AI systems. 
**Main Takeaways** 
* Reasoning has always been seen as a core characteristic of human intelligence.
Reasoning is used to derive new information from given base knowledge; this
new information is guaranteed correct when sound formal reasoning is used,
otherwise it is merely plausible.
* AI research has led to a range of automated reasoning techniques. These
reasoning techniques have given rise to AI algorithms and systems, including
SAT, SMT, and constraints solvers as well as probabilistic graphical models, all of
which play a key role in critical real-world applications.
* While large pre-trained systems (such as LLMs) have made impressive
advancements in their reasoning capabilities, more research is needed to
guarantee correctness and depth of the reasoning performed by them; such
guarantees are particularly important for autonomously operating AI agents 

**Context & History** 
Reasoning is a core component of
human intelligence. From the dawn
of humanity, abductive reasoning
has been used to predict danger and
inductive reasoning made it possible
to learn regularities governing the
world. Beginning in Ancient Greece,
deductive reasoning techniques were
developed to draw valid conclusions
that follow logically from premises
known to be true. The development
of reasoning methods with such a
priori guarantees was a key factor in
the advancement of modern science,
mathematics, and engineering; notably,
according to philosophers such as
Charles Sanders Peirce, the interplay
between abduction, deduction, and
induction forms the basis of the
scientific method and hence all modern
science. Attempts to mechanize
logical reasoning can be traced back to
13th-century philosopher Ramon Lull
and lie at the heart of the concept of
computation. Probabilistic reasoning
and inference have also profoundly
impacted reasoning, often relying on
the celebrated theorem by Thomas
Bayes on inverse probability that also
forms the basis for many machine
learning and statistics approaches.
Finally, the evaluation of correct (sound)
reasoning lies at the heart of most
quantitative assessments of human
cognition. 
Not surprisingly, reasoning has been
central to the AI enterprise. Indeed,
the earliest research in AI – from Logic
Theorist onwards [1] – had a strong
focus on reasoning [2]. Since the 1960s,
AI has also embraced probabilistic
reasoning and models, initially for
medical diagnosis [3]. Since then,
the reasoning tasks addressed in AI 
research have covered the gamut from
planning and temporal reasoning to
diagnosis and explanation. While early
AI has paid attention to both plausible
reasoning (case-based, analogical,
qualitative) and sound formal reasoning
with guarantees (logical, probabilistic,
constraint-based), over the years,
the focus has shifted more towards
reasoning with formal guarantees.
There are good reasons for this when
designing AI systems and techniques
that compensate for human limitations
and weaknesses since reasoning
with guarantees is challenging for
humans. This has led to practically
impactful applications of AI systems
such as SAT, SMT, and constraints
solvers, including the verification of
correctness properties of computer
hardware and software, the safety of
communications protocols, the design
of new proteins, and, more recently, the
robustness of neural networks against
adversarial attacks. It has also resulted
in probabilistic graphical models [4,
5], which are powerful modeling and
inference tools that have found their
way into numerous applications of
reasoning in medicine, robotics, and
beyond. 
**Current State & Trends** 
The emergence of the Internet and the
associated technology that made it
possible to capture the human digital
footprint at scale, as well as the leaps in
computing power, have made possible
novel approaches to learning bottomup from data. Of particular interest are
large pre-trained models, such as LLMs,
that have shown surprising abilities in
plausible reasoning. Unlike the earlier
research on reasoning in AI, LLMs
have focused on plausible reasoning patterns as they emerge automatically
after large-scale training on petabyte
corpora. While the results have been
quite remarkable so far, the reasoning in
this context has been of the “plausible”
variety with no guarantees. 
Meanwhile, sound formal reasoning
techniques remain key to important
and impactful applications of cuttingedge AI technology for the verification
of computer hardware and software,
as well as for real-world planning and
resource allocation problems. They
are also increasingly recognized as a
crucial basis for the formal verification
of machine learning techniques such
as neural networks, e.g., in the context
of local robustness against adversarial
attacks [6]. Significant research activity
takes place in these areas, focusing on
improving various types of reasoning
algorithms (notably with respect to their
computational complexity), leveraging
learning within sound formal reasoning,
and combining reasoning and learning
techniques [7, 8]. 
**Research Challenges** 
Bringing some of the rigorous a priori or
post hoc guarantees back into plausible
reasoning patterns turbocharged by
the pre-trained models has become an
active and promising area of research
– especially where AI systems need to
work autonomously in safety-critical
domains. Research on so-called “large
reasoning models” as well as on neurosymbolic approaches is addressing
these challenges. 
Furthermore, even though formal
reasoning with correctness guarantees
is currently considerably less in vogue
than the use of generative AI techniques
for plausible reasoning, formidable and essential challenges also remain in that
area. In this context, the combination
of machine learning techniques with
formal reasoning techniques holds
considerable promise for economically
and socially valuable breakthroughs,
notably in the area of AI safety and
transparency.
The questions and challenges we face
range from the philosophical:
* What exactly is “reasoning”?

to the practical:
* Can LLM ‘reasoning’ be trusted?

and include: 
What does the future hold for the
advancement and role of symbolic
reasoning?
* To what extent can LLMs or other
generative models reproduce or
replace symbolic reasoning?
* To what degree will symbolic
reasoning be necessary or sufficient
to overcome the current limitations
of LLMs?
* How well can AI reasoning,
especially LLM ‘reasoning,’ be
explained and understood?
* How can computers better
understand and simulate human
reasoning?
* What is the role of collaborative
reasoning between humans and
computers?
* How best can LLMs and symbolic
reasoning be integrated into “neurosymbolic reasoning”?
* Are further breakthroughs, beyond
both LLMs and traditional symbolic
reasoning, required to achieve AGIlevel reasoning?
* What forms of reasoning can best
support humans when dealing with
various challenges, e.g., in medical,
scientific, engineering, and legal
domains? 

**Community Opinion** 
The AAAI community appears to
strongly agree on the importance
of reasoning in AI systems. In our
community survey, slightly over 55%
of the respondents chose to answer
specific questions related to the topic
of reasoning. Of these, 79% indicated
that the topic of reasoning is relevant
to their research (with 44.7% marking
it as “very relevant”). Of the properties
required for referring to a process
as reasoning, 77.5% of the survey
participants marked “Knowledge can
be incorporated”, 72.5% “Explanations
can be provided,” and 56.9% “Involves
multiple steps to arrive at a conclusion”.
Interestingly, merely 37.4% indicated
“Guaranteed correctness of inference
results/outcomes”, and only 23.7% that
“A formal system and solver is used,”
which reflects the recent focus on
informal, plausible reasoning, likely in
the context of generative AI methods.
This suggests that an effort may be
warranted to better communicate the
importance and success of formal,
sound reasoning techniques. Finally,
44.7% of respondents agreed that
“Reasoning involves a search process.” 
There was broad agreement among
survey participants that focusing
reasoning research in AI on humanlevel reasoning is valuable (41.6%) or
even essential (47%); similarly, a focus
on domain-specific reasoning abilities
was seen by 49.6% of respondents as
valuable, and by 42.8% as essential.
This clearly reflects the importance
attributed to a research focus on
reasoning. 
The community also sees an exciting
potential of synergy offered by logical
and probabilistic models of reasoning
that were developed in AI prior to large
pre-trained models. This is clearly
reflected in the fact that 76.9% of survey
participants marked the integration
of learning and reasoning approaches
as very important (6 or 7 on a scale
of 7); interestingly, the percentage
of respondents that considered
Explainability and verifiability as very
important was similarly high (at 71.7%). 
Finally, 61.8% of survey participants
estimated the minimal percentage of
symbolic AI techniques required for
reaching human-level reasoning to be
at least 50% (with 24.8% estimating it
at 75% or more, compared to 38.2%
estimating it at 25% or below). What
remains unclear is the degree to
which AI researchers and practitioners
realize that decidedly superhuman
levels of reasoning are required for
and displayed in the prominent and
successful applications of formal AI
reasoning techniques for scientific
and mathematical discovery and
engineering applications, as well as in AI
safety. 
1. Newell, A. & Simon, H. (1956). The logic theory machine: A complex information processing system. IRE Transactions on Information Theory 2: 61-79. 
2. Brachman, R. and Levesque, H. (2004) Knowledge Representation and Reasoning (1st Ed). Morgan Kaufman. 
3. Russell, S. J., & Norvig, P. (2016). Artificial intelligence: a modern approach (4th Ed). Pearson. 
4. Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufman. 
5. Koller, D. and Friedmann, N. (2009) Probabilistic Graphical Models. The MIT Press. 
6. König, M. et al. (2024) Critically Assessing the State of the Art in Neural Network Verification. Journal of Machine Learning Research 25(12): 1-53 
7. Guo, D. et. al. (2025) DeepSeek-R: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. https://arxiv.org/abs/2501.12948 
8. Kambhampati, S. (2024). Can Large Language Models Reason and Plan? Annals of New York Academy of Sciences. March 2024.

AI Reasoning

Improving factually and trustworthiness of AI systems is the
single largest topic of AI research today, and while significant
progress has been made, most scientists are pessimistic that
the problems will be solved in the near future. 
**Main Takeaways** 
* An AI system is factual if it refrains from outputting false statements. Improving
factually of AI systems based on neural-network large language models is
arguably the biggest area of AI research today.
* Trustworthiness extends trustworthiness to include criteria such as human
understandability, robustness, and the incorporation of human values. Lack of
trustworthiness has always been an obstacle for deploying AI systems in critical
applications.
* Approaches to improving the factuality and trustworthiness of AI systems
include fine-tuning, retrieval-augmented generation, verification of machine
outputs, and replacing complex models with simple understandable models. 

**Context & History** 
A factual AI system does not output
erroneous information or hallucinate
answers. Before the era of generative
AI, problems with factuality arose when
systems were trained on bad data,
as captured by the slogan, “garbage
in, garbage out”. Work on methods
for improving data quality has a long
history in AI [1]. 
Generative AI, and in particular
large language models, employ
reconstructive memory - that is, they
rebuild memories as needed on the
basis of distributed bits of information
rather than retrieve memories from
a fixed store. The earliest generative
LLMs made an impact with their ability
to generate coherent but entirely
imaginary stories [2]. Factuality of LLMs
on a given domain was improved by
fine-tuning the model on domain data
[3]. 
Trustworthiness is a broader concept
than factuality because it includes
criteria such as understandability,
robustness, and respect of human
values. A traditional approach for
improving understandability of AI
systems is to replace complex blackbox models with simple humanunderstandable models - such as
naive Bayes [4] or generalized linear
regression [5]. Research on robustness
of machine learning studies how the
outputs of a model vary with small
changes in its training data. For
example, contrastive learning is a
method to train deep neural nets with
increased robustness [6]. Further
discussion of robustness in generative
AI appears in
this report’s section on Reasoning.
Discussion of respect of human values
by AI systems appears in many other
sections of this report and so will not be
discussed here. 

**Current State & Trends** 
As noted, fine-tuning remains the
main approach used by scientists
and engineers to improve factuality
of generative AI systems. In addition
to fine-tuning on domain-specific
documents, modern fine-tuning
includes reinforcement learning with
human feedback from thousands of
people. The cost of employing such
large numbers of human evaluators
is a major bottleneck for scaling AI
systems, and so there is much interest
in discovering methods to reduce the
amount of human feedback needed [7]. 
The second main technique for
improving factuality of generative AI
is retrieval-augmented generation
(RAG) [8]. In response to a question,
the system gathers a set of relevant
documents using traditional
information retrieval algorithms. The
AI system is then prompted to generate
an answer by combing through and
summarizing the retrieved documents.
While RAG can improve factuality, it
is dependent on the quality of data
retrieved. For example, if the target
document set is the entire web, it
can end up incorporating incorrect
information and even satirical stories in
its answer. 
Related to RAG is enabling the
generative AI system to use tools for
fact checking. Tools used by generative
AI systems include calculators, factual
databases such as citation indexes, and
formal planning and reasoning systems
[9]. A recent approach to improving
factuality is to provide the system with
a set of rules that state constraints on
space of answers. The output from the
model is verified against these rules and
inconsistent responses are culled [10].
Amazon Web Services already supports
this approach with “automated
reasoning checks” [11]. 
A third technique for improving
factuality of generative AI is chainof-thought (CoT), where a series of
prompts breaks down a question into
smaller units [12]. CoT often includes
steps where the model is asked to
reflect back on its tentative conclusions
and see if any are hallucinations. CoT
is discussed in more detail in the
Reasoning section of this report. 
The impact of data quality on factuality
was mentioned above. In addition
to fine-tuning on human curated
data, there is recent work on creating
synthetic data that is guaranteed to be
high quality for fine-tuning [13]. 
Trustworthiness, we noted, generalizes
factually and includes understandability
and robustness. One approach to
making neural network models more
understandable is to factor them into a
set of recognizers for high level features
and then combine the features using an
understandable model such as additive
regression [15]. Another approach is to
tease out how concepts and rules are
actually represented in a trained [16].
Understandability can also be
improved by employing CoT techniques
to ask a generative AI system to explain
the steps in its reasoning [17] or tell
the user when the system is uncertain
about a conclusion [18]. Finally, a
generative AI system can be asked not
to output a single answer, but instead
to distill a complex set of information into a simple human understandable
representation such as a decision tree
[19]. 
**Research Challenges** 
Factuality is far from solved. There
are a growing number of benchmark
dataset designed to test the factuality
of LLMs. One of the latest, SimpleQA
from Google, is a collection of simple,
unambiguous, timeless, and challenging
factual questions and answers
[14]. As of December 2024, the best
models from OpenAI and Anthropic
correctly answered less than half of the
questions. 
Robustness in generative AI can
be improved, as noted above, by
employing robust loss functions such
as contrastive learning. Adversarial
training, which applies perturbations in
the embedding space during training,
can improve both robustness and
generalization [20]. In addition, the
techniques for factuality generally
improve robustness as well. 
**Community Opinion** 
Over 75% of the AAAI community
strongly agreed that factuality and
trustworthiness were relevant or very
relevant to their own research. 
All six approaches mentioned in
the survey for improving factuality - external fact-checking tools,
reinforcement, improving data quality,
data curation, synthetic training, and
new neural net architectures - found
support. The greatest demand was
for more research on new neural net
architectures (73% marked important
or very important), followed closely by
external fact-checking tools (70%). 
For trustworthiness, new neural
net architectures were also viewed
as most important (77% important
or very important), followed by
enabling models to describe their
reasoning processes (70%) and use
of understandable models instead of
neural networks (61%). Notably, few
viewed research on giving AI systems
human-like personalities as important
to improving trustworthiness (24%).
Finally, the community mostly agreed
(59%) that trustworthiness as currently
formulated is ill-defined. Most also
disagreed (around 60%) that either
factuality or trustworthiness would
soon be solved. 
The AAAI community suggested
a number of additional aspects of
factuality and trustworthiness that were
not covered above. These included:
* The ability to understand and
present different sides of the same
issues, including pros and cons for
each one.
* Understanding that trustworthiness
depends upon the context of the
domain, organizational objectives,
and user objectives. It is wrong
to think of an AI system as simply
being trustworthy or untrustworthy
without regard to context.
* Transparency needs to go beyond
the models used to the actual
sources of training data. This should
include multi-source verification of
facts ingested by the models.
* The focus of work should be
on risk and mitigation rather
than on “solving” factuality and
trustworthiness.
* Attention needs to paid to giving
AI agents the ability to update
their knowledge while maintaining
reliability.
 

1. Budach, Lukas, Moritz Feuerpfeil, Nina Ihde, Andrea Nathansen, Nele Sina Noack, Hendrik Patzlaff, Hazar Harmouch and Felix Naumann (2022). The Effects of Data Quality on Machine
Learning Performance. https://arxiv.org/pdf/2207.14529
2. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI. Retrieved from https://cdn.openai.com/betterlanguage-models/language_models_are_unsupervised_multitask_learners.pdf
3. Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the
North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 4171–4186. https://doi.org/10.48550/
arXiv.1810.04805
4. Pedro Domingos & Michael Pazzani (1997). On the Optimality of the Simple Bayesian Classifier under Zero-One Loss. Machine Learning, 29(2-3), 103–130. https://doi.
org/10.1023/A:1007413511361
5. Caruana, Rich, Yin Lou, Johannes Gehrke, Paul Koch, M. Sturm and Noémie Elhadad (2015). Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day
Readmission. Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2015). https://people.dbmi.columbia.edu/noemie/
papers/15kdd.pdf
6. Hadsell, R., Chopra, S., & LeCun, Y. (2006). Dimensionality reduction by learning an invariant mapping. Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision
and Pattern Recognition (CVPR 2006), 2, 1735–1742. https://doi.org/10.1109/CVPR.2006.100
7. Hu, C., Hu, Y., Cao, H., Xiao, T., & Zhu, J. (2024). Teaching language models to self-improve by learning from language feedback. Findings of the Association for Computational Linguistics
(ACL 2024). Retrieved from https://aclanthology.org/2024.findings-acl.364/
8. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-augmented generation for
knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems (NeurIPS 2020), 33, 9459–9474. Retrieved from https://arxiv.org/abs/2005.11401
9. Guan, L., Valmeekam, K., Sreedharan, S., & Kambhampati, S. (2023). Leveraging pre-trained large language models to construct and utilize world models for model-based task
planning. Proceedings of the 33rd International Conference on Automated Planning and Scheduling (ICAPS 2023). Retrieved from https://arxiv.org/abs/2305.14909
10. Backes, J., Bolignano, P., Cook, B., Dodge, C., Gacek, A., Luckow, K., Rungta, N., Tkachuk, O., & Varming, C. (2018). Semantic-based automated reasoning for AWS access policies using
SMT. In 2018 Formal Methods in Computer-Aided Design (FMCAD) (pp. 1–9). IEEE. https://doi.org/10.23919/FMCAD.2018.8602994
11. Barth, Antje (2024). Prevent factual errors from LLM hallucinations with mathematically sound Automated Reasoning checks (preview). Posted 3 Dec 2024, retrieved 8 Feb 2025. AWS
News Blog, permalink https://aws.amazon.com/blogs/aws/prevent-factual-errors-from-llm-hallucinations-with-mathematically-sound-automated-reasoning-checks-preview
12. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022). Chain-of-thought prompting elicits reasoning in large language models. In Advances in
Neural Information Processing Systems (Vol. 35, pp. 24824–24837). https://proceedings.neurips.cc/paper/2022/file/9d5609613524ecf4f15af0f7b31abca4-Paper-Conference.pdf
13. Ding, Bosheng, Chengwei Qin, Ruochen Zhao, Tianze Luo, Xinze Li, Guizhen Chen, Wenhan Xia, Junjie Hu, Anh Tuan Luu and Shafiq R. Joty (2024). Data Augmentation using LLMs: Data
Perspectives, Learning Paradigms and Challenges. Annual Meeting of the Association for Computational Linguistics (2024). https://aclanthology.org/2024.findings-acl.97.pdf
14. Wei, Jason, Nguyen Karina, Hyung Won Chung, Yunxin Joy Jiao, Spencer Papay, Amelia Glaese, John Schulman, William Fedus (2024). Measuring short-form factuality in large language
models. https://doi.org/10.48550/arXiv.2411.04368
15. Agarwal, R., Melnick, L., Frosst, N., Zhang, X., Lengerich, B., Caruana, R., & Hinton, G. E. (2021). Neural additive models: Interpretable machine learning with neural nets. Advances in
Neural Information Processing Systems, 34, 2021. https://proceedings.neurips.cc/paper/2021/hash/251bd0442dfcc53b5a761e050f8022b8-Abstract.html?utm_source=chatgpt.com
16. Templeton, A., Conerly, T., Marcus, J., Lindsey, J., Bricken, T., Chen, B., Pearce, A., Citro, C., Ameisen, E., Jones, A., Cunningham, H., Turner, N. L., McDougall, C., MacDiarmid, M., Tamkin,
A., Durmus, E., Hume, T., Mosconi, F., Freeman, C. D., Sumers, T. R., Rees, E., Batson, J., Jermyn, A., Carter, S., Olah, C., & Henighan, T. (2024). Scaling Monosemanticity: Extracting
Interpretable Features from Claude 3 Sonnet. Transformer Circuits Thread. https://transformer-circuits.pub/2024/scaling-monosemanticity/index.html
17. Yeo, W. J., Ng, X. X., Le, T. K. C., & Lu, X. (2024). How interpretable are reasoning explanations from prompting large language models? Findings of the Association for Computational
Linguistics: NAACL 2024. Retrieved from https://aclanthology.org/2024.findings-naacl.138
18. Lin, S., Hilton, J., & Evans, O. (2022). Teaching Models to Express Their Uncertainty in Words. Transactions on Machine Learning Research. https://openreview.net/pdf?id=8s8K2UZGTZ
19. Chen, Y., Zhang, L., Wang, H., & Li, J. (2025). Zero-Shot Decision Tree Construction via Large Language Models. arXiv preprint arXiv:2501.16247. Retrieved from https://arxiv.org/
abs/2501.16247
20.Liu, X., Cheng, H., He, P., Chen, W., Wang, Y., Poon, H., & Gao, J. (2020). Adversarial training for large neural language models. arXiv preprint arXiv:2004.08994. Retrieved from https://
arxiv.org/abs/2004.08994

AI Factuality & Trustworthiness

AI & Sustainability

Artificial General Intelligence (AGI)

How should we challenge exaggerated claims about AI’s
capabilities and set realistic expectations?

**Main Takeaways**

* Over the last 70 years, against a background of constant delivery of new and important technologies, many AI innovations have generated excessive hype.
* Like other technologies these hype trends have followed the general Gartner Hype Cycle characterization.
* The current Generative AI Hype Cycle is the first introduction to AI for perhaps the majority of people in the world and they do not have the tools to gauge the validity of many claims 

**Context & History**

Artificial intelligence, or AI, is the field
that studies the synthesis and analysis
of computational agents that act
intelligently [6]. AI has gone through
hype cycles multiple times since the
1956 workshop that established the
name AI and set the course for early
computer science departments to
include AI as a major component of
their research and teaching. All hype
bubbles eventually burst, as the
essence of hype is that it is beyond
reality. Over the decades this has led to
AI winters where funding has dried up
for all of AI or for specific aspects of AI
such as neural networks or robotics.

A study published 2017 on trends with
the public perception of AI over a 30-
year period found that discussion about
AI had sharply increased since 2009
and that discussions in the public press
had been consistently more optimistic
than pessimistic [4]. The study also
found that hopes about AI applications
of AI in healthcare and education
were increasing over time. Another
finding was that concerns were growing
about loss of control of AI, ethical
implications, and the negative impact of
AI on work. 
Perhaps the difference in recent years
with prior periods is that hype has
gone beyond the pages of academic
conferences, conference papers, and
scientific magazines, out into both the
mainstream media, and social media.
AI and Artificial Intelligence have
become common words that nontechnical people have heard about
and a common subject for leaders
of almost all countries to talk about.
Governments, for the first time, have AI
policies. 
One of the problems is that AI is
actually a wide-reaching term that can
be used in many different ways. But
now in common parlance it is used
as if it refers to a single thing. In their
2024 book [5] Narayanan and Kapoor
likened it to the language of transport
having only one noun, ‘vehicle’, say, to
refer to bicycles, skate boards, nuclear
submarines, rockets, automobiles, 18
wheeled trucks, container ships, etc.
It is impossible to say almost anything
about ‘vehicles’ and their capabilities
in those circumstances, as anything
one says will be true for only a small
fraction of all ‘vehicles’. This lack of
distinction compounds the problem
of hype, as particular statements get
overgeneralized. 
The hype also sets expectations for
ordinary people. Many are fearful that
they will lose their jobs to AI in the
short term. Social scientists then work
to solve labor disruptions, e.g., for
displaced truck drivers [6], based on
predictions about AI (and in this case,
self-driving trucks) and its adoption that
turn out to be wildly optimistic. There
are no deployed self-driving trucks in
the predicted time frame. 
Hype in response to a technology
trigger is not restricted to AI. Indeed
the business intelligence company
Gartner, has deliberately made
a practice of using a graphical
representation of hype levels through
five stages that are common for many
technologies: (1) technology trigger,
(2) peak of inflated expectations, (3)
trough of disillusionment, (4) slope
of enlightenment, and (5) plateau
of productivity. They have used this
framework to track many technologies,
including quantum computers, block
chain, autonomous vehicles, nanotechnology, etc. In November 2024 they
[1] estimated that hype for Generative
AI had just passed its peak and was on
the downswing. 
The question for AI professionals is
how to respond to this hype, how to
question it, and how to help others
understand what is hubris, while
maintaining their own intellectual
modesty and probity. This is hard to do
in the middle of outsized claims about
one’s own field, and often it is up to
future historians to carefully dissect
past scientific arguments. 
Historian Thomas Haigh has tried to
do such a dissection, almost in real
time, in a recent series of articles in the
Communications of the ACM. In [2] he
gives a post-mortem on the impact of
over-hype in AI that resulted in what is
known as the AI-winter in the 1980s. His
one line summary is: “Fallout from an
exploding bubble of hype triggered the
real AI Winter in the late 1980s.” In [3] he
makes a comparison between the hype
of today and of those earlier times.
He summarizes this particular opinion
piece with the line: “From engines of
logic to engines of bullshit?” 
**Research Challenges** 
Many of us who have worked in AI for
decades face the challenge of trying to
remain honest brokers when we see
that many of the public statements of
people quite new to the field are out of
line with reality. 
The big question is whether, given
the dynamics of social media and the
search for clicks, professional opinions
and peer reviewed research papers
have any impact on dampening the
overclaims and the ways they distort
common understanding of where AI is,
and what is its potential in one year, five
years, ten years, etc. 
If we are currently left out of the
conversations how can we change that? 
**Community Opinion** 
The Community Survey gives
perspectives on the reactions to the AI
Perception vs Reality theme. First, the
results of the survey are summarized
here. 36% of the survey respondents
chose to answer the questions for this
theme. This is the summary breakdown
of the responses to each question: 
**1. How relevant is this Theme for your
own research?** 72% of respondents
said it was somewhat relevant (24%),
relevant (29%) or very relevant (19%). 
**2. The current perception of AI
capabilities matches the reality of AI
research and development.** 79% of
respondents disagreed (47%) or strongly
disagreed (32%). 
**3. In what way is the mismatch
hindering AI research?** 90% of
respondents agreed that it is hindering
research: 74% agreeing that the
directions of AI research are driven by
the hype, 12% saying that theoretical AI
research is suffering as a result, and 4%
saying that less students are interested
in academic research. 
**4. Should there be a communitydriven initiative to counter the hype
by fact-checking claims about AI?** 78%
yes; 51% agree and 27% strongly agree. 
**5. Should there be a communitydriven initiative to organize public
debates on AI perception vs reality,
with video recordings to be made
available to all?** 74% yes; 46% agree
and 28% strongly agree. 
**6. Should there be a communitydriven initiative to build and maintain
a repository of predictions about
future AI’s capabilities, to be checked
regularly for validating their accuracy?**
59% yes; 40% agree and 29% strongly
agree. 
**7. Should there be a communitydriven initiative to educate the public
(including the press and the VCs) about
the diversity of AI techniques and
research areas?** 87% yes; 45% agree
and 42% strongly agree. 
**8. Should there be a communitydriven initiative to develop a method
to produce an annual rating of the
maturity of the AI technology for
several tasks?** 61% yes; 42% agree and
19% strongly agree. 
Since the respondents to this theme
are self-selected (about a third of all
respondents), that bias must be kept
in mind. Of those who responded,
a strong and consistent (though not
completely monolithic) portion felt that
the current perception of AI capabilities
was overblown, that it had a real impact
on the field, and that the field should
find a way to educate people about the
realities.

1. Chandrasekaran, A. [2024]. What’s Driving the Hype Cycle for Generative AI, 2024. https://www.gartner.com/en/articles/hype-cycle-for-genai
2. Haigh, T. [2024]. How the AI Boom Went Bust. Vol. 67 No. 2, pp 22-26.
3. Haigh, T. [2025]. Artificial Intelligence Then and Now. Vol. 68 No. 2, pp 24-29.
4. Fast, E. and Horvitz, E. [2017]. Long-term trends in the public perception of artificial intelligence, AAAI’17: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence,
February 4, 2017, pp 963-969. https://ojs.aaai.org/index.php/AAAI/article/view/10635
5. Narayanan, A. and Kapoor, S. [2024]. AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference. Princeton University Press.
6. Poole, D. L. and Mackworth, A. K. [2023]. Artificial Intelligence: Foundations of Computational Agents, 3rd edition. Cambridge University Press.
7. Wang, S., Mack, E. A., Van Fossen, J. A., Medwid, M., Cotten, S. R., Chang, C. H., Mann, J., Miller, S. R., Savolainen, P.T. and Baker, N. [2023]. Assessing alternative occupations for truck
drivers in an emerging era of autonomous vehicles. Transportation Research Interdisciplinary Perspectives, Vol 19. May. https://doi.org/10.1016/j.trip.2023.100793

AI Perception vs. Reality

As AI becomes increasingly powerful and ubiquitous, it is disrupting skills and displacing workers. NSF’s National AI Institute for Adult Learning and Online Education (AI-ALOE) posits that AI can be part of the solution to the growing problem if we can use AI for reskilling, upskilling, and workforce development at scale. The long-term vision of AI-ALOE is to develop and use AI technologies to enhance the proficiency of online education for all adult learners, using in-person education as a benchmark. The day-to-day mission of AI-ALOE is to conduct responsible research into AI that is grounded in theories of human cognition and learning and derived from the scientific process of learning engineering. I will describe ongoing research at AI-ALOE.

AI for Reskilling, Upskilling, and Workforce Development

Small Data: A New Paradigm for the Next Generation of AI

An agent acts in its world to achieve its objectives. Intelligence allows the agent to make decisions and act. In natural domains, sensing is limited, so acting is gambling. It’s a myth that passive learning and more data are all we need. An agent cannot learn from observations alone. It needs a real body to carry out experiments in its world, testing hypotheses, to determine causation, refining its model of the world’s dynamics. The agent is acting as a scientist: refining its model through experiments and acting appropriately to achieve its objectives. Its objectives depend on its preferences and values, and those of other agents its actions impact. Determining which values to use, and how preferences can be acquired fairly, is a major non-technical challenge. We address three primary questions: What should an agent believe? What should an agent do, given its beliefs, preferences, and abilities? What should the preferences of an agent be? Integrating these issues motivates the design of our latest AI textbook, Artificial Intelligence: Foundations of Computational Agents, (3rd Ed. 2023).

Alan Mackworth is a Professor Emeritus of Computer Science at the University of British Columbia. He works on artificial intelligence with applications in constraint satisfaction, cognitive robotics, assistive technology, hybrid systems and constraint-based agents. He invented the world’s first soccer-playing robots. He has co-authored two books: Computational Intelligence: A Logical Approach (1998) and Artificial Intelligence: Foundations of Computational Agents (2023, 3rd Ed.). Alan co-founded the UBC Cognitive Systems Program, the Centre for AI, Decision-making and Action (CAIDA) and the AI network of BC (AInBC). He has served as President of AAAI, IJCAI and CAIAC. He is a Fellow of AAAI, CAIAC, AGE-WELL, CIFAR and the Royal Society of Canada.

David Poole is a Professor Emeritus of Computer Science at the University of British Columbia. He is known for his work on combining logic and probability, probabilistic inference, relational probabilistic models, statistical relational AI and semantic science. He is a co-author of two AI textbooks (Cambridge University Press, 3rd edition 2023, and Oxford University Press, 1998), and co-author of ” Statistical Relational Artificial Intelligence: Logic, Probability, and Computation”. He is a former chair of the Association for Uncertainty in Artificial Intelligence, the winner of the Canadian AI Association (CAIAC) 2013 Lifetime Achievement Award, and is a Fellow of the Association for the Advancement Artificial Intelligence (AAAI) and CAIAC.

The Essence of Intelligence is Appropriate Action (not thinking, reasoning, learning or language) and other things every student of AI should know

Professor Edward Feigenbaum: a Tribute to and Lecture by a Pioneer of AI on his 90th Birthday

We stand at the dawn of the AI era, a technological revolution poised to be the most consequential of our generation, presenting both unprecedented opportunities and profound challenges. But this promise is shadowed by significant challenges. To build a future we want, we must move beyond the hype and the headlines to confront the most pressing open problems—technical, sociotechnical, and multidisciplinary. This talk will review the rapid progress, dissect challenges ahead, and argue that our greatest task isn’t simply building smarter machines, but fostering the human wisdom to guide them towards a future that is not only intelligent but also equitable, safe, and profoundly human.

Navigating the AI Horizon: Promises, Perils, and the Power of Collaboration

Large language models have revolutionized textual reasoning, yet their ability to act meaningfully in multimodal, real-world environments remains limited. They struggle to ground their decisions in visual context, adapt to changing goals, and plan actions over time—shortcomings that stem from a lack of structured, goal-driven reasoning and insufficient representations of the physical world.

In this talk, I present a unified framework for building embodied agents that can see, simulate, and reason. I begin by introducing methods for learning world simulators from data, arguing that visual reasoning—like textual reasoning—benefits from step-by-step processing. Inverting a physics simulator becomes key: agents must infer structured 3D neural representations of objects, parts, motions, and scenes directly from raw video. I describe methods for extracting such representations using generative priors, injecting them into vision-language models (VLMs), and scaling up their supervision via generative 3D simulation and fast, modular physics engines. These simulators enable agents to anchor their predictions in grounded physical reality, reducing hallucinations and improving control.

Complementing this simulation capability, I explore techniques that enable agents to reason over time and adapt their behavior. By integrating structured memory systems, agents learn to retain and retrieve relevant experiences to inform long-horizon plans. Language-based reflective feedback allows them to refine their strategies beyond what sparse rewards offer, forming abstractions that generalize across tasks. When trained to ground their reasoning directly in the visual environment, agents gain the ability to set subgoals, explore effectively, and verify their own hypotheses.

Together, these advances point toward autonomous systems that simulate before they act, reflect after they fail, and maintain an ongoing awareness of goals, constraints, and context. I will illustrate these capabilities across web automation, robotics, and interactive assistance, showing how agents that see, simulate, and reason offer a promising path toward general-purpose embodied intelligence.

Towards Embodied Agents that See, Simulate, and Reason

From Workflows to Water Coolers: AI That Can Navigate Human Nature

Physicists aim to explain the Universe in terms of a compact, interpretable set of principles. Deducing those principles from experiments poses many challenging and problems which are ripe for application of AI and present opportunities to develop new AI techniques. I will describe how AI has changed the way particle physicists work and speculate about the role of AI in the future of fundamental physics. Finally, I will describe my experience in science communication, as an author, podcaster and television producer.

Fundamental physics and science communication

The pursuit of high-efficiency Artificial General Intelligence (AGI) requires more than brute-force scaling of model size and data. While scaling remains a key driver of capability, equally important are scalable architectural and principles—designs that continue to work, improve, and remain controllable as we vary model scale, domains, and modalities. Central to our approach is the concept of the “Specialized Generalist” – a pathway that achieves deep expertise across multiple domains without sacrificing broad generalization capabilities. In this talk, we introduce the “Specialized Generalist” paradigm and our implementation of it, SAGE (Synergistic Architecture for Generalized Expertise), a three-layer architecture designed to balance specialization and generalization in a systematic way. We will describe how SAGE’s Base Model, Synergy Fusion, and Exploration-Evolution layers interact in practice, focusing on concrete mechanisms for coordinating domain-specific expertise with broad general reasoning. We will share empirical results and recent advances in large reasoning models, embodied AI, and scientific applications to further illustrate the approach. A central motivation is to support “AGI for Science” by building a stable plateau of capabilities that can reliably assist with complex scientific workflows rather than isolated demos. Finally, we will outline the safety and governance questions that arise when deploying Specialized Generalist systems in high-impact settings, and discuss what we have learned so far about monitoring, alignment, and operational safeguards.

Quest of AI towards Specializable Generalist: From Reasoning to Scientific Discovery

There has been a lot of exciting recent progress on new and powerful machine learning algorithms and architectures: how to learn. But for autonomous agents acting in the dynamic, uncertain world, it is at least as important to be able to identify which concepts and subproblems to focus on: what to learn.
This talk presents methods for identifying what to learn within the framework of reinforcement learning, focusing especially on applications in multiagent systems and robotics.

From How to learn to What to learn in Multiagent Systems and Robotics

Today, search is used by LLMs to support retrieval-augmented generation as well as by humans. In this new setting, how does what we want from a search engine change, and what are promising approaches to take? I will describe work on two themes related to these questions: (1) retrieving documents to support complex information needs in the context of long-form retrieval-augmented generation and (2) exploring learned sparse retrieval as a promising paradigm for first-stage retrieval. In the former line of work, I will discuss search systems aimed at maximizing coverage of relevant information and approaches for evaluating their ability to do so independent of a generation system. In the latter line of work, I will describe research positioning learned sparse retrieval as a compelling alternative to dense retrieval.

Search with Complex Topics and Learned Sparse Retrieval

Language is one of the richest and most complex human cognitive capacities. Yet, we lack a model organism to study its underlying neural mechanisms: unlike other important cognitive capacities, such as vision or memory, language does not have a clear counterpart in non-human animals, leaving a gap in our ability to develop and test mechanistic hypotheses. In recent years, large language models (LLMs) have emerged as the closest computational analogs we have but how can we use them effectively as model organisms for language in the human brain? In this talk, I will discuss the promise and challenges of this approach. I will present our recent work on brain-tuning using naturalistic brain recordings to refine LLMs so that their internal representations and processing better align with human neural data. But beyond representational similarity, a key question remains: do LLMs rely on mechanisms that are similar to those in the brain? And at what level of abstraction should we assess this similarity? This research direction aims to transform LLMs from mere engineering artifacts into powerful scientific tools for uncovering how the brain supports our most distinctive cognitive ability.

Large Language Models as Model Organisms of Language in the Human Brain

Communication across languages has been historically a "holy grail" in the context of AI. In a world with more than 7000 languages, AI still falls short in terms of coverage or equitably serving different languages. For decades we have focused on high-resource languages, since models have largely been resource-dependent. Today we are witnessing the evolution of techniques that can contribute to better represent a larger number of languages. We are kind of breaking the “digital desert”. In this talk, we will cover an overview of the path towards Omnilinguality in AI through the lens of the Machine Translation, starting with basic but complex definitions such as language or long-tail. MT is one of the most popular and well-explored applications in multilinguality. MT has been shown to scale to a massive number of languages with flagship specialised models covering from hundreds up to thousands of languages. Progress in MT has been largely driven by the scientific community effort of WMT that has put together competitions, benchmarks and constantly posing new challenges. Other key contributions have been the open-sourced evaluation datasets such as FLORES that cover hundreds of languages in the Wikipedia domain, and more recently BOUQuET that builds on top of non-English data, originally created and expanding to more domains. These initiatives are complemented with open-initiatives that allow the community to contribute with more languages. In my personal opinion, while MT has served as a platform to reach such progress, it may not always be the main one. Even if we have not yet solved this task, the rise of LLMs shows the opportunity to solve many tasks at once. MT will still be a relevant part of the puzzle by being a source of synthetic data generation and a representative evaluation in breadth. However, I will argue that Omniliguality should not be faced task-specifically anymore and it should be taken as a great arena to test more broadly advanced LLMs techniques and an opportunity to come up with new methods to massively scale general purpose evaluation.

Omnilinguality, Scaling AI to Any language

Over the past four decades, work at the intersection of Arabic and technology has evolved alongside major technical breakthroughs and shifting regional and global political dynamics. In this talk, I revisit the development of Arabic NLP and AI, from early foundational efforts to today’s large-scale generative systems, and highlight how each phase has engaged with distinct aspects of Arabic’s linguistic complexity. I reflect on the language’s ambiguous orthography, rich morphology, diglossic landscape, and wide geographic and historical reach, and on the growing research community committed to ensuring that AI technologies meaningfully support Arabic and its diverse cultural expressions. I conclude with a forward-looking vision for building a cohesive and sustainable ecosystem that advances Arabic in AI through strengthened training, deeper collaboration, and sustained innovation for the next generation of researchers and practitioners.

Arabic and Technology: A 40-Year Perspective

Recent years have witnessed the rise of increasingly larger and more sophisticated language models (LMs) capable of performing every task imaginable, sometimes at (super)human level. In this talk, I will argue that there is still space for specialist models in today’s NLP landscape. Such models can be dramatically more efficient, inclusive, and explainable. I will focus on two examples, opinion summarization and crosslingual semantic parsing and show how these two seemingly unrelated tasks can be addressed by explicitly learning task-specific representations. I will show how such representations can be further structured to allow search and retrieval, evidence-based generation, and cross-lingual alignment. Finally, I will discuss why we need to to use LLMs for what they are good at and remove the need for them to do things that can be done much better by smaller models.

Prompting is not all you need! Or why Structure and Representations still matter in NLP

That the recent LLM breakthroughs are solely due to scaling is a myth. Many difficult research problems had to be solved to make models like GPT4 and Mixtral possible. One of those difficult research problems is data quality. Data quality is a great challenge for NLP researchers with many opportunities for innovation and impact on current generative AI developments. I will focus on two examples in my talk: quality data for training a highly multilingual language model and quality data for instruction tuning.

Quality Data for LLMs: Challenges and Opportunities for NLP

**Karen Spärck Jones Award Lecture** 
The advent of generative AI technology produces transformative impact on the content creation landscape, offering alternative approaches to produce diverse, good-quality content across media, thereby reshaping the ecosystems of online content creation and publishing, but also raising concerns about market over-saturation and the potential marginalization of human creativity. Our recent work introduces a competition model generalized from the Tullock contest to analyze the tension between human creators and generative AI. Our theory and simulations suggest that despite challenges, a stable equilibrium between human and AI-generated content is possible. Our work contributes to understanding the competitive dynamics in the content creation industry, offering insights into the future interplay between human creativity and technological advancements in generative AI.

Human vs. Generative AI in Content Creation Competition: Symbiosis or Conflict?

Language Use in Embodied AI

Chatbots for Good and Evil

Going beyond the benefits of scale by reasoning about data

Vision-Language Models (VLMs) often struggle to balance visual and textual information when summarizing complex multimodal inputs, such as entire TV show episodes. In this paper, we propose a zero-shot video-to-text summarization approach that builds its own screenplay-like representation of an episode, effectively integrating key video moments, dialogue, and character information into a unified document. Unlike previous approaches, we simultaneously generate screenplays and name the characters in zero-shot, using only the audio, video, and transcripts as input. Additionally, we highlight that existing summarization metrics can fail to assess the multimodal content in summaries. To address this, we introduce MFactSum, a multimodal metric that evaluates summaries with respect to both vision and text modalities. Using MFactSum, we evaluate our screenplay summaries on the SummScreen3D dataset, demonstrating superiority against state-of-the-art VLMs such as Gemini 1.5 by generating summaries containing 20% more relevant visual information while requiring 75% less of the video as input.

Integrating Video and Text: A Balanced Approach to Multimodal Summary Generation and Evaluation

Document Visual Question Answering (DocVQA) models often produce overconfident or ethically misaligned responses, especially under uncertainty. Existing models like LayoutLMv3, UDOP, and DONUT focus on accuracy but lack ethical calibration. We propose HonestVQA, a model-agnostic, self-supervised framework that aligns model confidence with correctness using weighted loss and contrastive learning. We introduce two new metrics—Honesty Score (H-Score) and Ethical Confidence Index (ECI)—to evaluate ethical alignment. HonestVQA improves accuracy and F1 by up to 4.3% across SpDocVQA, InfographicsVQA, and SROIE datasets, while reducing overconfidence. It also generalizes well across domains, achieving 78.9% accuracy and 76.1% F1-score.

The Confidence Paradox: Can LLM Know When It’s Wrong?

State space models (SSMs) are a promising alternative to transformers for language modeling because they use fixed memory during inference. However, this fixed memory usage requires some information loss in the hidden state when processing long sequences. While prior work has studied the sequence length at which this information loss occurs, it does not characterize the types of information SSM language models (LMs) tend to forget. In this paper, we address this knowledge gap by identifying the types of tokens (e.g., parts of speech, named entities) and sequences (e.g., code, math problems) that are more frequently forgotten by SSM LMs. We achieve this by training an auto-encoder to reconstruct sequences from the SSM's hidden state, and measure information loss by comparing inputs with their reconstructions. We perform experiments using the Mamba family of SSM LMs (130M--1.4B) on sequences ranging from 4--256 tokens. Our results show significantly higher rates of information loss on math-related tokens (e.g., numbers, variables), mentions of organization entities, and alternative dialects to Standard American English. We then examine the frequency that these tokens appear in Mamba's pretraining data and find that less prevalent tokens tend to be the ones Mamba is most likely to forget. By identifying these patterns, our work provides clear direction for future research to develop methods that better control Mamba's ability to retain important information.

Characterizing Mamba's Selective Memory using Auto-Encoders

Relational reasoning lies at the core of many NLP tasks, drawing on complementary signals from text and graphs. While prior research has investigated how to leverage this dual complementarity, a detailed and systematic understanding of text-graph interplay and its effect on hybrid models remains underexplored. We take an analysis-driven approach to investigate text–graph representation complementarity via a unified architecture that supports knowledge co-distillation (CoD). We explore five tasks involving relational reasoning that differ in how text and graph structures encode the information needed to solve that task. By tracking how these dual representations evolve during training, we uncover interpretable patterns of alignment and divergence, and provide insights into when and why their integration is beneficial.

R²-CoD: Understanding Text-Graph Complementarity in Relational Reasoning via Knowledge Co-Distillation

The success of Transformer language models is widely credited to their dot-product attention mechanism, which interweaves a set of key design principles: mixing information across positions (enabling multi-token interactions), sequence-dependent activations (where attention weights adapt to each input), a specific mathematical form (dot-product similarities plus softmax weighting), and coupling of queries and keys to evolving hidden states (grounding attention in the current layer). However, the necessity of each of these principles remains largely untested. In this work, we systematically deconstruct attention by designing controlled variants that selectively relax these principles, applied both uniformly across all layers and in hybrid architectures where only some layers retain standard attention. Our empirical analysis reveals that mechanisms for mixing tokens are indispensable, as their absence collapses models to near-random behavior, while the exact mathematical form and sequence dependency can be substantially relaxed, especially when preserved in just a subset of layers. Surprisingly, even variants that fail in isolation can achieve robust performance when interleaved with standard attention, highlighting a cooperative effect. These findings deepen our understanding of what truly underpins attention's effectiveness and open new avenues for simplifying language models without sacrificing performance.

Deconstructing Attention: Investigating Design Principles for Effective Language Modeling

Generating rational and generally accurate responses to tasks, often accompanied by example demonstrations, highlights Large Language Model's (LLM's) remarkable In-Context Learning (ICL) capabilities without requiring updates to the model's parameter space. Despite having an ongoing exploration focused on the inference from a document-level concept, its behavior in learning well-defined functions or relations in context needs a careful investigation. In this article, we present the performance of ICL on partially ordered relation by introducing the notion of inductively increasing complexity in prompts. In most cases, the saturated performance of the chosen metric indicates that while ICL offers some benefits, its effectiveness remains constrained as we increase the complexity in the prompts even in presence of sufficient demonstrative examples. The behavior is evident from our empirical findings and has further been theoretically justified in term of its implicit optimization process.

The code is available \href{https://github.com/DDuttaGit/ICLonPartialOrder}{here}.

Assessing the Limits of In-Context Learning beyond Functions using Partially Ordered Relation

Although behavioral studies have documented numerical reasoning errors in large language models (LLMs), the underlying representational mechanisms remain unclear. We hypothesize that numerical attributes occupy shared latent subspaces and investigate two questions: (1) How do LLMs internally integrate multiple numerical attributes of a single entity? (2) How does irrelevant numerical context perturb these representations and their downstream outputs? To address these questions, we combine linear probing with partial correlation analysis and prompt-based vulnerability tests across models of varying sizes. Our results show that LLMs encode real-world numerical correlations but tend to systematically amplify them. Moreover, irrelevant context induces consistent shifts in magnitude representations, with downstream effects that vary by model size. These findings reveal a vulnerability in LLM decision-making and lay the groundwork for fairer, representation-aware control under multi-attribute entanglement.

Interpreting Multi-Attribute Confounding through Numerical Attributes in Large Language Models

In the deployment of Large Language Models (LLMs), ``spurious correctness''---where answers are correct but reasoning contains errors---poses a critical risk by creating an illusion of reliability. While prior work on LLM confidence estimation focuses on answer-level or entire reasoning path confidence, these coarse-grained approaches fail to identify which specific parts of the reasoning contain errors. We propose a fine-grained confidence estimation framework that computes confidence scores for individual evidence triplets within reasoning chains, enabling precise localization of errors. Using carefully designed prompts, we generate answers, evidence in triplet format, and their respective confidence scores simultaneously, allowing automatic detection of spurious correctness patterns where partial evidence contains factual errors. Evaluated on both Japanese and English multi-hop QA benchmarks across multiple models from three model families representing different architectures and training approaches, we show that our approach exhibits superior calibration performance for evidence confidence and demonstrates effective ability to detect spurious correct answers (up to 0.84 on our primary discrimination metric). The consistent improvements across languages demonstrate the generalizability of our method. As a secondary benefit, joint generation of confidence scores improves answer confidence calibration by up to 43\%. This prompt-based approach requires no model retraining and is immediately applicable to existing LLMs.

Fine-grained Confidence Estimation for Spurious Correctness Detection in Large Language Models

Multilingual generation with large language models (LLMs) is often of poor quality for mid- to low-resource languages, but the causes for this are not well-understood. We first demonstrate the existence of an implicit task-solving→translation pipeline for generation, whereby the model first solves the required task in a largely target-language-agnostic manner, and subsequently translates answer concepts into the intended target language. We hypothesize that the failure of the translation stage, despite task-solving success, is an important culprit for the observed low quality of final outputs, and formalize this as the translation barrier hypothesis. We quantify the extent to which either stage in the pipeline is responsible for final failure for a word translation task across 108 language pairs, and find that the translation barrier explains a dominant portion of error for a majority of language pairs, and is especially severe for low-resource target languages. Our results highlight an important bottleneck for end-to-end multilingual generation, relevant for future work seeking to improve multilinguality in LLMs.

The Translation Barrier Hypothesis: Multilingual Generation with Large Language Models Suffers from Implicit Translation Failure

Pruning techniques have been studied to construct small models for efficiency, yet the effect of cross-lingual, which shows language performance transferability, is understudied in this field. In this work, we investigate cross-lingual effects in multilingual large language model compression using iterative pruning and recovery. We employ structured layer pruning with LoRA-based recovery and knowledge distillation, testing whether calibration languages different from target evaluation languages can preserve multilingual performance. Experiments on Qwen2.5-7B and Llama3.1-8B demonstrate that any recovery language consistently outperforms no-recovery baselines, with even low-resource languages like Swahili providing ~5% improvements. In contrast to expectations, dominant pretraining languages do not always yield the best results, where Indonesian achieves the highest performance in Llama3.1-8B, while Japanese performs the best in Qwen2.5-7B. Our findings reveal that cross-lingual calibration effectively maintains multilingual capabilities in the iterative pruning.

Multilingual Iterative Model Pruning: What Matters?

We explore whether synthetic datasets generated by large language models using a few high quality seed samples are useful for low-resource named entity recognition, considering 11 languages from three language families. Our results suggest that synthetic data created with such seed data is a reasonable choice when there is no available labeled data, and is better than using entirely automatically labeled data. However, a small amount of high-quality data, coupled with cross-lingual transfer from a related language, always offers better performance. Data and code available at: https://github.com/grvkamath/low-resource-syn-ner.

Does Synthetic Data Help Named Entity Recognition for Low-Resource Languages?

Language model architectures are predominately first created for English and afterwards applied to other languages. This can lead to problems for languages that are structurally different from English. We study one specific architectural choice: positional encodings. We do this through the lens of the trade-off hypothesis: the supposed interplay between morphological complexity and word order flexibility. This hypothesis states there exists a trade-off between the two: a more morphologically complex language can have a more flexible word order, and vice-versa. Positional encodings are a direct target to investigate the implications of this hypothesis in relation to language modelling. We pre-train and evaluate three monolingual model variants with absolute, relative and no position encodings for seven typologically diverse languages and evaluate on four downstream tasks. We fail to find a consistent trend with various proxies for morphological complexity and word order flexibility. Our work shows choice of tasks, languages, and metrics are essential for drawing stable conclusions.

On the Interplay between Positional Encodings, Morphological Complexity, and Word Order Flexibility

Despite remarkable advances in natural language processing, developing effective systems for low-resource languages remains a formidable challenge, with performances typically lagging far behind high-resource counterparts due to data scarcity and insufficient linguistic resources. Cross-lingual knowledge transfer has emerged as a promising approach to address this challenge by leveraging resources from high-resource languages. In this paper, we investigate methods for transferring linguistic knowledge from high-resource languages to low-resource languages, where the number of labeled training instances is in hundreds. We focus on sentence-level and word-level tasks. We introduce a novel method, GETR (Graph-Enhanced Token Representation) for cross-lingual knowledge transfer along with two adopted baselines (a) augmentation in hidden layers and (b) token embedding transfer through token translation. Experimental results demonstrate that our GNN-based approach significantly outperforms existing multilingual and cross-lingual baseline methods, achieving 13 percentage point improvements on truly low-resource languages (Mizo, Khasi) for POS tagging, and 20 and 27 percentage point improvements in macro-F1 on simulated low-resource languages (Marathi, Bangla, Malayalam) across sentiment classification and NER tasks respectively. We also present a detailed analysis of the transfer mechanisms and identify key factors that contribute to successful knowledge transfer in this linguistic context.

BhashaSetu: Cross-Lingual Knowledge Transfer from High-Resource to Extreme Low-Resource Languages

Temporal reasoning and knowledge are essential capabilities for language models (LMs).
While much prior work has analyzed and improved temporal reasoning in LMs, most studies have focused solely on the Gregorian calendar.
However, many non-Gregorian systems, such as the Japanese, Hijri, and Hebrew calendars, are in active use and reflect culturally grounded conceptions of time.
If and how well current LMs can accurately handle such non-Gregorian calendars has not been evaluated so far.
Here, we present a systematic evaluation of how well language models handle one such non-Gregorian system: the Japanese *wareki*.
We create datasets that require temporal knowledge and reasoning in using *wareki* dates. 
Evaluating open and closed LMs, we find that some models can perform calendar conversions, but GPT-4o, Deepseek V3, and even Japanese-centric models struggle with Japanese calendar arithmetic and knowledge involving *wareki* dates.
Error analysis suggests corpus frequency of Japanese calendar expressions and a Gregorian bias in the model's knowledge as possible explanations.
Our results show the importance of developing LMs that are better equipped for culture-specific tasks such as calendar understanding.

Can Language Models Handle a Non-Gregorian Calendar? The Case of the Japanese wareki

Large language models (LLMs) provide detailed and impressive responses to queries in English. However, are they really consistent at responding to the same query in other languages? The popular way of evaluating for multilingual performance of LLMs requires expensive-to-collect annotated datasets. Further, evaluating for tasks like open-ended generation, where multiple correct answers may exist, is nontrivial. Instead, we propose to evaluate the predictability of model response across different languages. In this work, we propose a framework to evaluate LLM's cross-lingual consistency based on a simple Translate then Evaluate strategy. We instantiate this evaluation framework along two dimensions of consistency: information and empathy. Our results reveal pronounced inconsistencies in popular LLM responses across thirty languages, with severe performance deficits in certain language families and scripts, underscoring critical weaknesses in their multilingual capabilities. These findings necessitate cross-lingual evaluations that are consistent along multiple dimensions. We invite practitioners to use our framework for future multilingual LLM benchmarking.

Found in Translation: Measuring Multilingual LLM Consistency as Simple as Translate then Evaluate

Multimodal sentiment analysis identifies human emotional tendencies by analyzing text, visual, and auditory modalities. In most studies, the textual modality is usually considered to contain the most emotional information and is regarded as the dominant modality. Existing methods mostly map auxiliary modalities into a semantic space close to the dominant modality, which overly relies on the dominant modality. In this work, we propose a Feature Decomposition-Augmentation (FeaDA) framework, which aims to elevate the role of auxiliary modalities in multimodal data fusion. We first design a projector to decompose auxiliary modalities into partial features, which contain features for emotion judgment, and then utilize these decomposed features to guide the fusion process with KL loss, thereby enhancing the status of auxiliary modality fusion. To verify the effectiveness of our method, we conducted experiments on the CMU-MOSI, CMU-MOSEI, and CH-SIMS datasets. The experimental results show that our FeaDA framework outperforms mutilmodal sentiment analysis methods of the same type in main metrics. Our code is available at https://github.com/PowerLittleYin/FeaDA-main.

Underline digital video library

Browse The Future of AI Research Webinars

Browse EACL Keynotes and Invited Talks

Browse Underline Originals

No results!

Browse AAAI 2026 Invited Talks

Browse Natural Language Processing

Stay up to date with the latest Underline news!

PRODUCT

COMPANY

RESOURCES