Using Reinforcement Learning to Run a Cleaner Bot

Reinforcement learning can be used to program a cleaner bot to clean the floor. The example given here uses the Markov Decision Process and code in Python.

- Neha Ray

Reinforcement learning (RL) interacts with a dynamic environment and learns from the consequences of its actions. This is different from supervised learning, where the model is trained on a labelled dataset, or unsupervised learning, where the model looks at unlabelled data and tries to find hidden patterns or structures on its own.

A cleaner bot is a classic 'toy problem' used to show how an autonomous agent should make decisions under uncertainty. This simplified model of an automated robotic vacuum cleaner balances its functions between two conflicting goals:

• Maximising work done (cleaning floors)

• Minimising risk (running out of battery)

As the actions of the cleaner bot have uncertain outcomes, the Markov Decision Process (MDP) is used to describe the environment. The outcomes of the actions are as follows:

• The reward: It gets points for every square metre cleaned.

• The cost: Every movement consumes battery.

• The risk: If the battery hits zero while it's far from the dock, it dies (a large negative reward).

An example: The office vacuum cleaner

Imagine a robot vacuum cleaner is assigned to clean a wide lobby in an office. It has the following three possible states and three possible actions.

The three states are:

• High battery: The robot is fully charged.

• Low battery: The robot is at 15% battery and the warning light is on.

• Charging: The robot is docked at the charging point.

The three actions are:

• Search: Drive around and clean (high reward, high battery consumption).

• Wait: Stay still and save energy (no reward, no battery use).

• Recharge: Move back to the charging point (no reward, restores battery).