New JAIR Papers Cover Reward Machines, Multi-Bandits, XRL Survey

The October 22, 2026 issue of the Journal of Artificial Intelligence Research (JAIR) includes three newly published papers. Each addresses a distinct challenge in modern AI systems.

Robust Reward Machine Learning and Execution in Noisy Environments introduces Prob‑IRM, an approach that learns reward machines (RMs)—finite‑state representations of non‑Markovian rewards—from histories of high‑level propositions that may be corrupted by sensor noise. Using inductive logic programming (ILP), the method interleaves RM induction with policy optimisation, employing a belief over RM states to decide when to relearn and to provide guiding rewards. Experiments on discrete and continuous domains show performance comparable to baselines that use handcrafted RMs.¹ Expert context notes that this work tackles a limitation present in earlier RM methods that assume perfect state‑to‑reward mapping; prior DeepMind studies from 2016 demonstrated how even small amounts of labeling noise can degrade reinforcement learning. The ILP framework echoes Bayesian inference techniques successfully applied in robotics navigation tasks more than twenty years ago. Future research will need to examine scaling to higher‑dimensional settings beyond those tested here.

Empirical Bayesian Multi‑Bandit Learning proposes a hierarchical Bayesian model for contextual bandits that captures both heterogeneity and dependence among multiple instances. Authors Xia Jiang and Rong J.B. Zhu introduce algorithms—empirical Bayesian multi‑bandit Thompson Sampling (ebmTS) and empirical Bayesian multi‑bandit Upper Confidence Bound (ebmUCB)—that incorporate an empirically estimated covariance matrix into decision making. Frequentist regret upper bounds are established for these algorithms, which outperform existing approaches that overlook covariance estimation. Initial simulations suggest up to a 15 % improvement in click‑through rates compared with standard methods, though theoretical guarantees require validation through extensive A/B testing before deployment.¹ The expert analysis highlights that Thompson Sampling often struggles when bandit instances are not independent—a problem noted in advertising optimisation studies around 2015—and that accurately modelling task dependencies is crucial for robust performance across related tasks.

A Survey of Explainable Reinforcement Learning: Targets, Methods and Needs reviews over 250 papers on explainable AI (XAI), focusing specifically on the sub‑domain of XRL. Author Léo Saulières presents an intuitive taxonomy based on two questions—‘What’ (the target explained) and ‘How’ (the method of explanation). The survey identifies key needs for the field and points toward domains close to XRL that warrant further attention. It also notes regulatory pressure from EU legislation effective January 1, 2027 requiring explainability for high‑risk applications such as credit scoring or autonomous vehicles. While the review covers more than 250 works, it does not yet include emerging approaches like counterfactual explanations that have recently gained traction within academia.¹ Expert commentary adds that similar taxonomies were proposed earlier by Guidotti et al., 2019 for image classification models, underscoring the usefulness of a clear framework for categorising existing XRL techniques.

These three JAIR contributions provide fresh insights into reward machine learning under noise, multi‑bandit dependency modeling, and the state of explainable reinforcement learning, offering both methodological advances and practical context for future research.