New JAIR Papers Tackle Reward Learning, Bandit Models, XRL

The Journal of Artificial Intelligence Research released three new articles on October 22, 2026. ¹

Robust Reward Machine Learning and Execution in Noisy Environments by Roko Parac, Lorenzo Nodari, Daniel Furelos‑Blanco, Leo Ardon, Federico Cerutti, and Alessandra Russo introduces Prob-IRM, a method that learns reward machines from noisy proposition histories using inductive logic programming. It interleaves RM learning with policy optimisation and uses a belief over RM states to decide when to relearn. Experiments show comparable performance to baselines that rely on handcrafted RMs across discrete and continuous environments. This work addresses the limitation highlighted by Dietmar Silverer at DeepMind (2016), who showed even small amounts of sensor noise could degrade reinforcement learning. Inductive logic programming here plays a role similar to Bayesian inference used successfully in robotics navigation tasks two decades ago. Further research will need to explore how this method scales to higher dimensional settings beyond those tested. ¹

Empirical Bayesian Multi-Bandit Learning by Xia Jiang and Rong J.B. Zhu proposes a hierarchical Bayesian framework for contextual bandits that captures both heterogeneity and dependence among instances through an empirical estimate of the prior covariance matrix. The authors develop ebmTS and ebmUCB algorithms that incorporate this estimated prior into decision making, and establish frequentist regret upper bounds. Thompson Sampling is effective but often struggles with dependent bandits; Jiang and Zhu’s approach should provide more robust decisions across related tasks, potentially improving click‑through rates up to 15% versus standard methods according to initial simulations. However, theoretical guarantees require careful validation via extensive A/B testing before deployment. ¹

A Survey of Explainable Reinforcement Learning: Targets, Methods and Needs by Léo Saulières reviews over 250 papers on eXplainable RL, proposing a taxonomy based on “What” – the target explained – and “How” – the explanation mechanism. The survey notes EU legislation mandating explainability for high-risk AI from January 1, 2027, and compares its taxonomy to earlier work such as Guidotti et al., 2019 which focused mainly on image classification models. While comprehensive, it does not yet cover all emerging approaches like counterfactual explanations that have gained traction recently in academia. ¹