LabVIEW Robotics Documents

cancel
Showing results for 
Search instead for 
Did you mean: 

NIWeek 2010 Robotic Swarm Demo: Data Communications

This page is part of the NIWeek 2010 Robotics Demo series of blog posts.  For an introduction to the demo and links to other parts of the series, visit http://decibel.ni.com/content/docs/DOC-13031/.

Overview and Requirements


The unique requirements of robotic systems often require that communication between parts of robotic systems be unique to the application in which it is used. Some attempts at standardization exist, but so far none have been both widely adopted and generally accepted as a standard.  In the case of our NI Week Robotic Swarm demo, we had the following considerations to guide our design:

  • We could not guarantee that one or any of the robots would connected at any given time, for example, because we needed to change their batteries periodically.
  • We needed to be able to easily debug our code.
  • Multiple robots must connect to a network and communicate wirelessly.

The next section discusses how we addressed the first two concerns. The Choosing a Wireless Technology section will discuss our solution for the third concern.

Architecture

The fact that our demo is a swarm, with multiple independent robots, motivated us to break up our code into modules that can start and stop independently of each other. That is, having seperate modules that do not depend on other modules running allows for a module to "drop out" without communication issues. For example, when we power off a single robot to change its batteries, we don't want communication to the other three robots to be interrupted.

Breaking code into modules and defining a communication protocol with which they talk to each other also made it easier to monitor and debug the code for a couple reasons. First, the resulting code left logical places in the communication process at which to look at the data in the code—those places where the modules talk to each other. Think of this as similar to eavesdropping on a conversation. Second, because a module can run locally or on the network, you can debug code on either your desktop, on an RT target, or in simulation, without having to change anything.

You can find our data communication code at (path as viewed in NIWeek 2010 Robotics Demo.lvproj) My Computer > Data Communications. You'll quickly notice that our communication code can be a lot to wade through, so to make things a little easier to find, we've marked VIs that we reference in this document in the screenshot that follows.

File Directory.png


To dig a little more into what our code for data communication looked like, our solution consists of one server and multiple client nodes that connect to the server, where a client node could be a robot, a robot driver station, a machine running our localization routine, a machine displaying video feeds of the demo, a debugging client, and so on. Each client node has a unique ID (see Table 1). You can find the code for the user interface to our server at (path as viewed in NIWeek 2010 Robotics Demo.lvproj):

My Computer > Data Communications > NI_Robotics_Server UI.lvlib > Server UI.vi

To actually perform communication tasks, each client node contains a messaging client that communicates with the server over TCP. Messaging clients are created in VIs such as the one at the following location (path as viewed in NIWeek 2010 Robotics Demo.lvproj):

My Computer > Data Communications > Messaging Clients > NI_Robotics_Messaging Client > Create Messaging Client.vi

Once created, the messaging client for each client node performs the following tasks:

  • Listens for heartbeat messages from the server to monitor the status of the connection.  In the absence of a heartbeat message, the messaging client tries to reconnect to the server.
  • Sends and receives messages directly to and from the server.

On the server side of our code, for each client node that tries to connect to the server, the server generates a connection handler.  A connection handler performs the following tasks:

  • Sends heartbeat messages and any other TCP messages directed to its client.
  • Receives all TCP messages from that client node.

If the connection to a client node is lost, the connection handler terminate s.

Also on the server side, an arbiter determines which messages to send to which particular connection handler/client pair. We originally developed arbiters to simulate the limited communication range that typically affects swarm robotics applications. Often, transmitting messages from robot-to-robot is limited by the distance between robots and whether the robots have line of sight.  As a result, the arbiter allows our robots to communicate only to robots that are nearby.  If two robots are far apart, they must relay messages through another robot.

The figure that follows illustrates the relationships between client nodes, messaging clients, connection handlers, and the arbiter.

NIWeek Communications Architecture.png

Also notable about the arbiter is that it handles client subscriptions. Each client node can subscribe to other client nodes by specifying the IDs of the clients it wishes to subscribe to.  The general implementation of the code that initializes client subscriptions is at My Computer > Data Communications > Messaging Clients > NI_Robotics_Messaging Client > Create Messaging Client.vi.  A more advanced example, robots subscribing to other clients, is in the VI at My Computer > Data Communications > Messaging Clients > NI_Robotics_Robot Messaging Client.lvclass > Create Robot Messaging Client.vi.

Once a client node connects to the server, its messages are passed on to the clients subscribed to its ID. For example, consider a scenario where Robot #1 subscribes to the localization client node to receive updated information about its current location in the robot arena. However, the localization client has not started yet, so Robot #1 does not receive any position information.  Later, when the localization routine starts, its messaging client connects to the server, and the server, which passes the GPS messages along to the robots, transmits the localization messages to Robot #1 through its client subscription.

The following table lists and describes the client nodes that make up our demo.

Table 1. Client Nodes

Node ID

Description

0

Communications server

1-19

Robots

21-38

Driver stations—Each driver station can control any number of robots.

40

LCD display—UI that runs on the server machine and displays video feeds from each robot, a map of the robot arena, and so on.

41

Demo operator User Interface—UI that controls which robots are connected to which base station slots,  what modes (e.g. autonomous, tele-op) they operate in, and more.

43

Localization—Calculates and broadcasts the location of the robots.

44+

Any additional clients

Message Handling

In our demo, every message sent between client nodes is its own class. You can find these messages in the LabVIEW project at My Computer > Data Communications > Message Classes. By using classes to define messages, we can create a generic "Message Type" from which all other messages inherit.  In other words, both an Obstacle Position Message and a Robot Position Message are a "Message Type."  With this inheritance taking place, we only need to create communication code that handles "Message Type," and not each non-generic message. Further, if we add want to add a new message, like our Emergency Stop Message, we don't have to change our communication code to accomodate it as long as Emergency Stop Message inherits from "Message Type."

As a messaging client receives messages over TCP, the messages enter the client node's message queue. You can find much of the code for messaging clients at (path as viewed in NIWeek 2010 Robotics Demo.lvproj):

My Computer > Data Communications > Messaging Clients > NI_Robotics_Messaging Client > Messaging Client.vi

A few notable features of our messaging clients include:

  • Messages can have a high priority flag. The client recognizes the flag and responds by adding the message at the front of queue. For example, we want to process an Emergency Stop Message before a Robot Position Message, so we set the high priority flag for the former.

  • The code for each messaging client includes the VI at My Computer > Data Communications > Message Clients > NI_Robotics_Messaging Client.lvclass > Message Handlers > Incoming Message Handler Loop.vi which dequeues messages and, based on the type of message, updates corresponding value(s), such as robot position, the state of the mapped arena, and goal positions to which robots will navigate, by reference using data value references. We create new data value references at the same time we create message clients, then access them later using the In Place Element structure and the appropriate border nodes. This feature is important to note because by updating the value by reference, other processes also running do not have to wait for a message to arrive in order to execute. Consider the following example:

    1. The messaging client receives a GPS message and adds it to the queue.
    2. The messaging client receives an obstacle message and adds it to the queue.
    3. The message handler dequeues the GPS message and updates the robot position by reference.
    4. The message handler updates the obstacle information by reference.

Any process that uses the robot’s position will use the new position data as soon as the message handler updates the value by reference. Until then, processes can keep using the old data. Updating values by reference can be tricky, especially if you do not want to act on old data. Therefore we include a timestamp with every message sent so that processes can decide to discard obsolete data or wait until new data arrives.

  • Processes running on each client add messages to the client's outgoing message queue. For example, robots have a path planning loop that adds messages to the outgoing queue in its messaging client and a rangefinder loop that also adds messages to the outgoing queue (more on the code for path planning and rangefinding in a later post).  We pass the queues for clients throughout our code as references, so multiple processes, like the path planning and rangefinding loops in the previous example, can run in parallel and add messages to the same queue. Then, each messaging client has a loop, labeled outgoing message loop in My Computer > Data Communications > Messaging Clients > NI_Robotics_Messaging Client > Messaging Client.vi, that dequeues outgoing messages and sends them over TCP to the server.

Allowing Control by a Master Client

Our demo operator client contains a user interface with which a single person can control the behavior of every robot and dictate which driver stations are permitted to control the robots. For example, if your robotics application includes the ability to teleoperate robots, you can use this demo operator to stop a robot from operating autonomously and give control of the robot to a user at a driver station. You can find the code for our Demo Operator UI at (path as viewed in NIWeek 2010 Robotics Demo.lvproj):

My Computer > Data Communications > NI_Robotics_Demo Operator UI.lvlib > Demo Operator UI.vi


All of the robots subscribe to messages from the demo operator client. A few features of the demo operator code are that it can perform the following tasks:

  • Assign control of any or all robots to a driver station.
  • Change the operation mode of each robot. Operation modes include autonomous, semi-autonomous, and teleoperate, which we'll explain in an upcoming post.
  • Stop the robots from moving, and then resume their movement.
  • Stop the rangefinder from panning, and then resume its motion.
  • Clear all of the map data built up by the robots and displayed in the driver user interface, another topic we'll discuss more later.

Choosing a Wireless Technology

To arrive at a solution for communication between all the clients in our demo, we ran through a series of options, all of which we'll explain in this section.

Problem #1 As mentioned before, our demo consisted of multiple robots with wireless connections that needed to exist on the same network as several other clients.

Solution #1 Gaming adapters, which are often used to connect a video game console to a Wi-Fi network to enable Internet or LAN gaming, could allow the Single-Board RIO on each of our robots to connect wirelessly to a network.  Essentially, the gaming adapter converts a wired Ethernet connection to a wireless connection.

Problem #2 While gaming adapters can be a simple solution to networking multiple robots together, they do not have the ability to connect more than one wired network device to a wireless network.  This became a limitation when we outfitted each of our robots with an IP network camera (Axis M1011).  An IP camera can be accessed from any computer and any LabVIEW target with LabVIEW Vision software that is on the network.  So, with two network devices that each need a wireless connection (the sbRIO and the camera) on the same robot, a gaming adapter was not sufficient.

Solution #2 A wireless router could solve Problem #2 since it would allow both the sbRIO and the IP camera to connect to its network.  Then, users could wirelessly connect to the router and use the sbRIO and camera.

Problem #3 A wireless router was a perfectly acceptable solution for one robot.  However, in the case of our multiple robots, we would have to connect to each robot’s wireless network, and then somehow share the networks with each other.

Solution #3 A better solution is a wireless network bridge (see figure below), which is a sort of cross between a router and a switch.  We wanted the router to establish a connection with some base wireless network, and then share that network connection to devices we plug into it.  There are many commercially available solutions that accomplish this, but they are often aimed at business solutions and are typically expensive. This brought us to DD-WRT.

DD-WRT is an open-source firmware suitable for many different wireless routers.  Most home routers do not have wireless bridging as an available mode of operation.  However, a wireless home router formatted with DD-WRT can operate as a wireless bridge, or in other network configurations.

The following image illustrates how a wireless bridge can link multiple devices on the same wireless network. See http://www.dd-wrt.com for more information about compatible routers and how to set up a wireless bridge, and check out our previous work with DD-WRT and the Robotics Starter Kit at Teleoperate the Starter Kit from 20 floors Away.

http://www.dd-wrt.com/wiki/images/thumb/3/3a/Standard_bridge_large.jpg/798px-Standard_bridge_large.jpg

We first used the ASUS GT32 wireless router, which although compatible with DD-WRT, required a little extra work when setting up the open-source firmware.  This router is 2.4GHz 802.11/a/b/g router with four Ethernet ports. The router worked well in our office, connecting all of the robots to a wireless hub and allowing the camera feeds to stream across the network (albeit at low quality and low frame rates due to the limited bandwidth).

Problem #4 Fortunately for us, we had the foresight to also test our networking solution at the Austin Convention Center, where the demo would take place.  Unfortunately for us, we quickly found that the Convention Center’s wireless network crushed our setup.  Many times we could not even connect to our wireless hub with our laptops.  When we were able to connect, we saw ping times timeout or be as high as 2000-4000 ms, and the robots were never able to reliably stay connected to the wireless hub.  Also, because of security issues, the Convention Center WiFi did not allow us to form a network bridge.  This last issue prevented us from using the robots’ routers to connect to the Convention Center WiFi and use that as the wireless hub.

Solution #4 Seemingly at an impasse with our setup, we discussed using a 5GHz 802.11n router.  As it turned out, the 5GHz band at the Convention Center was not saturated like the 2.4GHz band.  In addition to being able to establish communications, the 5GHz  routers also gave us a much higher rate of data transfer, which meant that we could stream much higher quality video from the IP cameras at much higher frame rates.  This resulted in a much a better teleoperated feed for the driver and even allowed us to do some more advanced vision processing techniques and implement our goal-finding algorithm (more on all of these subjects in a later post). A less desirable effect of swapping the original routers for the Netgear WNDR3300 we decided to use instead was that we had to spend $100 on each robot’s router instead of just $30 (prices as of 7/15/2010).

Comments
Member technion
Member

i have been learning this demo project for 3 weeks by now and i have many questions since i design a similar project with 2-3 robots and a an experiment controller implemented on a pc.

my first question is what is the hirerchy between the massage class and what is the perpose of each class? for example there is a massage class and aclient massage class what is the diffrence?

second, the data comm in this project is implemeted mainly on the host side and mant host class does not have an eqvivalent on the robot side and i dont understeand how the robot can handle these massages.

third it seems like there are  massage queues (incomming and outgoing) but i couldnt find any link of these queues to any tcp\ip or wifi vis, so how the comm is done after adding massage to a queue?

i have some more questions regarding the adaptations i have to do in order to run on my project so i need some advise.

Contributors