Robust Reward Machine Learning from Noisy Sensors

The Journal of Artificial Intelligence Research published a study titled “Robust Reward Machine Learning and Execution in Noisy Environments” on October 22, 2026. It was written by Roko Parac, Lorenzo Nodari, Daniel Furelos‑Blanco, Leo Ardon, Federico Cerutti, and Alessandra Russo.

The paper introduces Prob‑IRM, an algorithm that challenges the long‑standing assumption that reward machine learning requires perfect state-to-label mappings. In practice, sensors can be faulty or incomplete; prior demonstrations by Dietmar Silverer at DeepMind in 2016 showed even modest noise levels could cripple reinforcement learning performance.

Reward machines encode non‑Markovian reward functions as finite-state automata over high-level propositional events. Traditional approaches rely on deterministic labeling functions that map environment states to proposition sets, but real-world labeling functions are inherently noisy. Prob‑IRM replaces this requirement with inductive logic programming (ILP). By treating noisy proposition histories as training examples, ILP learns an RM that maximises coverage of low‑uncertainty histories while interleaving RM learning with policy optimisation.

This belief-driven approach also decides when to relearn the RM and supplies additional guiding rewards, allowing the agent to adapt its understanding of the reward structure on the fly.

The authors evaluated Prob‑IRM across environments featuring both discrete and continuous state spaces under varying degrees of labeling noise. Results show that Prob‑IRM performs comparably to baselines that use handcrafted RMs—a significant achievement given it does not require manual specification of the reward machine.¹

Expert commentary notes that Prob‑IRM addresses a critical limitation of traditional reward-machine learning, which assumes perfect knowledge about how sensor data maps to propositions. Similar techniques using Bayesian inference have been successfully applied in robotics navigation tasks for two decades; further research will need to explore scaling Prob‑IRM to higher-dimensional environments beyond those tested here.