Accessibility Blog

From guide dogs to guide robots

Written by Rafal Charlampowicz, Contact him at LinkedIn

This article is not about digital accessibility. It’s about guide robots and guide dogs and sometimes even about human guides. I hope it will be useful for robot designers  and interesting for people who would like to know more about guide dogs, guiding and navigation.

Why don’t we have guide robots yet?

In Japanese and Chinese hospitals there are guide robots for sighted patients, but there are no guiding machines for blind people. There are some prototypes and some of them are promising, but there is no final product. Why? I have a thesis, which of course, could be wrong. A long time ago I saw (heard) a video showing a prototype of a guide robot which was walking down and up the stairs. The robot was walking extremely slowly, but at that time it was still a significant engineering achievement. While preparing for this article I came across news about a guide robot that can recognize traffic lights and signs. This is all fine and great, but this may be the actual source of the problem. Both features (stairs and traffic lights) are useful, but I would rather have a robot that does not walk the stairs and does not recognize traffic lights than have a robot that does both things but does not actually exist. My thesis is that engineers often try to achieve too much; whereas most people need solutions for basic, common problems. For example, guide dogs do two things: they avoid obstacles and recognize objects to which they guide. If the robot could do the same, it would be a great guide; better than most dogs.

Contemporary guide dogs’ training is based on over a century of experience and many solutions to well-known navigation problems can be copied from the dogs. I’m also going to discuss what is crucial and what is just nice-to-have in a guide robot.

Shape and user position

I will not recommend what is better: wheels or legs. Engineers are aware of the advantages and disadvantages of both solutions. Previously, I thought that wheels were better, but my recent experience with a quadruped robot at ETH in Zurich convinced me that legs are good as well. Some people believe that a robot that resembles a dog may be socially more acceptable, because people are used to guide dogs and their role. I think that the role of a canine shape should not be overestimated. People, e.g. on the street, may be more quick to react to a guide robodog, but there are other ways to draw attention to a guide robot, such as text or a signaling white cane.

If the robot is to be used exclusively in an indoor environment and will not travel with a user (for instance it is provided specifically for use in a hotel, university, office, etc.) and it can walk the stairs (or there are no stairs), then its size does not matter. It should fit the door and not block corridors and be safe for one’s feet. A noise produced by the robot may be important depending on the place it is to be used.

As regards robots for outdoor usage, (which will always implies indoor usage), designers should keep in mind that the robot must be put somewhere on a bus, tram and train. A robot that is not mobile or compact enough will not be used. It is not crucial that the robot is able to walk the stairs, however if it doesn’t, it must be light enough to be carried. The user may also need to carry the robot when it runs out of energy. While guiding the robot should slightly pull the user), but it does not have to move independently. I’ve recently heard about a prototype that you can push, but I haven’t a chance to test it yet, so I cannot say whether this passive movement is also good. Of course, it is useful if the robot can come to your call like guide dogs do, but this is not a must-have feature. I can easily imagine a robot that has the shape of a cane with a wheel and does not move independently (for example, it must be held by the user). There is even a prototype that has two wheels and a grip.

While talking about shape one should take into consideration the position of the user in reference to the robot. There are three options:

• In front of the robot
• Next to the robot
• Behind the robot

The first option may look absurd, but there are people who guide blind people by holding their arm or elbow. This way of guiding is very uncomfortable and dangerous and makes one feel like an object.

Two other options are applicable and each solution has its pros and cons. When guided by a guide dog or by a human, a blind person stands next to the guide (usually on the right side of the guide). In the case of a guide dog, the dog’s hind paws are next to the legs of the guided person. In the case of a human guide, a guided person usually walks half a step behind the guide. The advantage of this position is that one immediately feels the movement of a dog or a person and can quickly react. This reaction (for example, adaptation to a change of terrain) is sometimes not even conscious. For instance, by the movement of your guide you feel that the pavement is uneven, or that there is a level change etc. In the case of dogs, the additional advantage of this position is that the handler may control the dog with a leash. The guide dog guides with a harness. The handler holds the handle of the harness and feels the dog’s movement. The leash, which is normally held with a harness in the left hand, is for controlling the dog. Whenever additional control is needed, the handler takes the leash into the right hand and uses it to control the dog.

Another option is walking behind a guide. This is not normally done with dogs and humans (unless one person guides many blind people), except for narrow paths where there is no place on a path to walk next to your human guide, such as on a train or while hiking. As regards a robot, this user position is probably technically less demanding (and “less” is usually “better”) and may be useful in narrow areas like corridors and crowded places. The potential disadvantage is that when the robot suddenly stops, the user has little time to react. Problems may also occur on stairs. While going down the user shouldn’t be pulled. The robot must also be able to take 90 degree turns on the stairs while having the user standing behind it. All these problems can of course be solved. Designers must just be careful.

Repeated tasks

Before I go into detail as far as guiding is concerned, it is worth highlighting one thing that is often misunderstood.  The robot does not need to be a universal guiding machine. There are two types of blind people: people who walk alone and people who will not walk alone. People who walk alone usually go along the same routes. This is commonly a route to work and back, or a route to another frequently visited place. Rarely would you go to a place you know nothing about. Orientation is a quite demanding task and walking alone is often stressful and requires a high level of attention. If you make a robot that can guide someone along a pre-recorded route, you will significantly improve the lives of many people. Universality is good, but it won’t be a problem if, let’s say, the robot is not able to guide me from platform 1, because it was taught to guide from platform 2. In this instance, it is important that the robot lets me know it is not able to guide me and that it is able to avoid obstacles and guide me in the direction I command, or it is mobile enough to be pulled or carried through the part of the path it does not know.

People who currently do not walk alone may take advantage of guide robots, but one shouldn’t expect this. The reasons why many blind people do not move independently are usually due to psychological reasons and even the best guide may not convince someone to go out by themselves. Still, the better tools we have, the greater the likelihood this will change.

Avoiding obstacles

As I wrote in the introduction, guide dogs do two things: avoid obstacles and recognize objects. Dogs, except for frequent routes, do not know the destination. It is the handler who must know where to go and gives commands to the dog. But dogs are very good at avoiding obstacles. The robot must be the same. It should avoid obstacles on the route, and it should move along the track that is walkable. Generally, this means that the robot should avoid the street or grass etc unless it is clearly commanded to go onto these areas. If the robot does not go along a pre-recorded route, it should keep to the direction of movement, avoiding obstacles while moving. If it is not able to avoid obstacles, it should inform the user. Guide dogs often can get lost in crowded or complex areas. The dog’s handler is then not able to give the dog a command, because he or she does not know the correct direction and the dog does not know where to go. This is one task that is likely to be easier for robots than for dogs and their owners.

Recognizing objects

One great advantage of guide dogs is their ability to recognize objects. A list of commands the dog knows is often different depending on their school. Of course, the dog can learn new commands, such as finding a door, a street crossing, a free seat, stairs or an elevator; these are common across schools. In addition, my dogs could also identify bus stops, shop assistants and clerks, cars and even vending coffee machines (I trained my dog to do the latter on my own). With AI tools, robots could identify even more objects and guide users to them. The list of objects a guide robot recognizes could be much broader than in the case of guide dogs and you can find any such list in any assistive app that recognizes and describes objects. However, there are basic objects that are important for navigation.

List of objects important or useful while navigating:

• Crossing – the dog guides the owner to the nearest street crossing (the dog recognizes zebra crossings) and stops just before it. The dog’s head is directed towards the crossing.
• Door – the dogs guides the owner to the nearest door or entrance. This can refer to any kind of door including trains, buses, and other means of transportation.
• Bus stop – the dog guides the owner to the nearest place that looks like a stop: bus stop, tram stop or similar.
• Stairs – the dog guides the owner to the nearest stairs.
• Stairs up / down – the dog guides the owner to stairs that go up or down. This is very useful when stairs go in both directions.
• Elevator – the dog guides the owner to the elevator. This may be difficult to implement in a robot, because the dogs seem to sniff out the elevator. They actively search for it and do not limit the search to what can be seen from where they are currently located.
• Chair – the dog guides the user to a free seat (on a train, bus, building or a bench in a park – any free seat).
• Free space – in a crowded area, e.g. in a corridor, the dog tries to find a free space. Usually, it means that the dog guides the owner to the nearest wall. Only one of my dogs could find free space, but it was very useful to me.
• Shop – the dog guides the owner to the nearest kiosk or other small shop. When it comes to robots it would be best if the robot could recognize any location mentioned by the owner and guide them there.
• Car – the dog guides the owner to the nearest car. The robot should be more specific and differentiate between a passenger car, a bus and a minibus. Such differentiation is important when various kinds of vehicles may stop in the same area (e.g. at an airport).
• Person – dogs are not trained to search for a person, but robots should be able to guide users to the nearest person or a person selected by the owner. This is very important when you need to ask someone either for information or for help.

When it comes to robots the list of recognized objects has no real limits. For instance, recognizing room numbers and other text would be very helpful and shouldn’t be too difficult to implement. Recognizing friends and family members would also be helpful.

In fact, I have many ideas, but first let’s get a working robot and then let’s add extra features! For most of the features guide dogs are of course, the best reference, because their training is based on years of experience. However, it is important to remember that robots do not have the same limitations of the dogs.

Skills

In addition to avoiding obstacles and finding places and objects guide dogs have some skills which should be replicated by guide robots; however this may be sometimes difficult.

Dogs’ skills:

• Follow – the dog follows a person or a group of people. This is very useful when you walk with a group. You don’t need to give any commands to the dog, because it is following someone from the group (and the owner can relax). Dogs usually are also able to walk next to someone so that you are not behind, but side-by-side to your companion (this is useful when you are accompanying just one person).
• Guide on your own – the dog is allowed to break the rules to some degree to avoid a complex obstacle. For instance, if there is a large puddle that covers a whole sidewalk, the dog may walk to the grass (or to a street if needed) and bypass the obstacle. This command has various names, e.g. “forward around” or “bypass” and sometimes is split into two: passing by the obstacle without breaking rules and free guiding. The differences are subtle, and some schools may not agree to any breaking of the rules.
• Exit – the dog finds an exit from the building. This skill is very important and is not limited to looking for the door. The dog tries to find the entrance through which the team (dog and its owner) entered the building. It is worth mentioning that for a blind person finding an exit from a complex building may be a difficult, stressful and time-consuming task.
• Find – the dog looks for an item lost by the user. This usually refers to a situation when you drop something. You may ask a dog to find it (this may be rather difficult to implement in a robot, but at the same time, is not crucial for navigation).

On the road: commands and behaviour

Rules followed by guide dogs are also a good reference for designing robots’ behavior. However, they are not perfect, because robots may also be able to describe the area to the user – one thing dogs are not able to do. This often leads to misunderstandings between the dog and its owner. I recommend copying these rules unless tests show something different. As regards navigation commands, again, they are based on years of experience, so they well reflect the basic needs of blind people who walk with a dog.

Rules followed by guide dogs:

• Don’t start moving (guiding) without a command.
• Stop when told to stop (or stopped by a leash).
• Stop at an obstacle that cannot be easily avoided. This is of course partly subjective because the dog decides. The dog for instance will stop at a big hole in a pavement but may guide its handler through a set of footbridges over holes at a construction works.
• Guide along the best path, e.g. go along the side of the sidewalk that is least crowded. Unfortunately, sometimes dogs do this and sometimes they don’t.
• Don’t move even when commanded if you see it is dangerous. This so-called “smart disobedience” is incredibly important, because the dog’s owner may incorrectly judge the situation and order the dog to walk onto a busy street, walk off a platform edge, walk under a tram, etc. In the case of robots perhaps the best solution would be if a robot stops in a dangerous place and describes the situation. Then it should be up to the user if he or she wants to continue. The command to continue should be different from the standard “go” or “forward” command to avoid any misunderstandings. Defining dangerous situations by a robot may be a challenge. At the very least sudden changes in the height of the ground should be included.
• Stop before street. This refers to any crossing of a street, whether there is zebra crossing or not. The dog stops and starts moving when ordered by the owner. This rule also applies to internal roads that cross the sidewalk and even garage exits. In reality some handlers teach their dogs to ignore internal roads if there is no real traffic. As regards robots, all this depends on the technology. If the robot can reliably assess whether the street may be safely crossed, then there is no need for stopping, otherwise it is always better to stop. Dogs stopping at crossings also have an informative value. A blind handler can count crossings and learn where she or he is. With robots and/or GPS apps, the user can easily get current location information with no need for counting anything.
• Stop at turns. The dog should stop at sidewalk intersections and other possible turns. With robots, I don’t think this is useful unless the user does not have any information about the area.
• Don’t enter or exit a bus, train or any other means of transportation without an order. With technology where it currently is, it is likely to be easier to carry the robot in and out than giving any orders.
• Stop at stairs. This command particularly refers to going down stairs. The dog stops before the stairs and the handler is to feel (with a foot or a cane) the edge and make the first step.

And I would like to add my own rule which I taught all my dogs and which saved my nape a few times!

• Do not move down the stairs even when ordered to do so. When you are in a hurry and you are an experienced guide-dog handler, and your dog stops suddenly, it’s easy to think that it stopped without a clear reason. If you then command the dog to move forward (because you think it stopped in error) the dog might pull you down the stairs. With robots I would implement the same solution I suggested for dangerous situations. And of course, a robot may actually be able to announce why it has decided to stop moving.

Commands

• Forward – The dog starts guiding.
• Back – The dog makes an 180 degree turn and guides back.
• Left/right – The dog turns left/right. This command is sometimes split into “left/right” and “left turn/right turn”, where the latter means the dog should turn at the nearest turn, whereas the former means that the dog turns 90 degrees immediately.
• Straight – The dog moves to the central part of the sidewalk.
• To the left/right – The dogs guides along the left or right side of the sidewalk or path. This is useful, for example, when there is a bicycle lane on the right or left side of the sidewalk.

Note: these commands may have different names in various schools and not all of them may be taught in each school.

Interface

The one thing that shouldn’t be copied from guide dogs is the communication interface. A team in which one cannot see and one neither can speak nor write, will always have communication problems (which is a significant source of stress for handlers and dogs – this fact is usually ignored in materials describing the work of a guide dog). The robot should allow basic operations with no need of an app. By basic operations I mean stopping the robot and resuming its movement. The user should be able to select the pre-recorded route without additional tools. Where using an app is necessary, the user must be allowed time to put his or her phone away before the robot starts moving. It is also crucial that a user always have access to information about battery level. For other applications using it with a single headset is fine; many blind persons use a headset to hear their GPS navigation. The robot should have a speaker, but the truth is that in an urban environment, a speaker is rarely useful even when located over an ear – cities are just too noisy. Applications, if digitally accessible, have an advantage that users can not only perform complex tasks with their help, but their interface is also accessible for people with both people with vision and hearing problems. A speaker implemented within a robot will not be useful for this group of users. At the same time a speaker may be useful (for those who can hear) to find the robot. The robot must be able to produce a sound on demand so that it can be located. Of course, if the robot is perfect, it is able to find the user on its own, but even then it should be able to bark, i.e. beep when needed.

Conclusion

Guide dogs can do many things. They guide their owners by avoiding obstacles, they recognize various objects and guide to them, they follow the rules that make independent moving of a blind person as safe as possible. Stopping at dangerous and difficult places, stopping at street crossings, waiting for a command to get on or off the train or bus are good examples of such rules. These skills and rules should be implemented in guide robots, because guide dog training is based on many years of experience and real-life problems that blind walkers encounter.

Guide dogs also have significant limitations. They cannot describe the area, there are communication problems, their training may not be good enough, and last, but not least, they are living creatures who are not immune to stress. Guide robots may be much better guides than guide dogs. Actually, they may be the first tool that can replace the cane and the dog.  However, it is very important that designers focus on the most important problems of blind people who walk independently, instead of trying to solve all the problems. The robot does not have to be an universal guide. Initially, it is enough if it allows safe navigation on pre-recorded routes, keeps direction, and may be carried or moved in places it is not able to guide. Even such a limited guide will be helpful because its user will solve other problems on her or his own. People who walk alone are used to solving problems. If the robot solves just these navigational problems (e.g. the ability to move quickly on the route you know but which is full of obstacles which slow you down), then it will be a success.