Sunday, January 27, 2013

Remotely Operated Rover - Software Design and Networking

Once I had the IOIO working from the sample application, I started looking into how to control it across the network. Fairly early on, I needed to decide if the phone was going to act as the server, or the client. I figured there were advantages each way:

- If the phone was the server, it would be pretty easy to allow it to accept connections  from a client anywhere on the internet. My home router could pass the connection through to the port the phone was listening on, and you'd be off and running.

- If the phone was the client, it would be easier to make work over the 3G/4G cell network. It would reach out using whatever network connection it had (cell, wifi, either one) and connect to a fixed server, which would control it. It would be difficult or impossible to use the cell network if the phone was running the server, since the carriers almost certainly have firewalls in place.

I eventually settled on option one.. the phone would be the server. A client would connect to it and recieve sensor data, and issue commands to the phone to use actuators on the rover.

Next came a client, and control protocols. I wanted something fast - sensor data and control commands should be sent promptly to give a smooth driving experience. That ruled out a simple CGI/webserver sort of arrangement.  A web browser could act as a client only if something like Ajax was used to stream the commands. It would be slick to do that - a client computer likely already had a web browser. But I decided against it on several grounds:

- I don't know anything about Ajax. Like, nothing at all.
- It would complicate writing the server
- I wanted something as fast as possible, and I figured I could write a tighter custom protocol that would require less parsing

So I settled on a simple socket server running on the phone, and tested it with a Telnet client on Linux. I figured a proper client program could come later.

With that decision made, I started looking at socket programming in Android and Java. I had never done sockets before, so it was a good opportunity to learn some client/server stuff. It became apparent that I would need to break my network communications off into another thread to avoid blocking the server when it was waiting for input. If you happen to be using a GUI, this is particularly important since your GUI will stop responding during the blocking operation. Android will actually clobber your app if it blocks the GUI thread for more than a fraction of a second to ensure a good user experience.

So I decided on three threads:

1) The "master" thread, based on the HelloIOIOService example code
2) The IOIO looper thread
3) The network communication thread.

Threads in Java are actually pretty easy to spawn off, and child threads can access variables and functions of the parent. I am fairly new to Java, but it's not difficult using examples on StackOverflow and tutorial sites. It was fairly easy to spawn a thread, open a socket, listen on a port, and then exchange data with the master thread.

The tricky part was to make it work reliably. I wanted a continuous stream of sensor data and commands flowing between the client and server, a few times a second. So several issues you run into are:

1) You need to properly handle a client politely disconnecting and wait for new connections.
2) You need to handle a client just vanishing, rudely, detect it, and set up for when the client comes back.
3) You need to update the master thread so it can tell the IOIO looper what to do.
4) You need to properly handle the network dropping out from under the whole mess.

The first two proved challenging. I ran into a problem where the client could connect, and begin bouncing data back and forth, and run for several minutes. It would then crash. I eventually figured out that the Droid X2 has a weird problem - when running version 2.3.5, it will disconnect from my WiFi every 5 minutes or so, for about 5 seconds. I flashed it with a new 2.3.5 ROM called Eclipse, which is very nice. The problem persists, however. I'm convinced that it's an issue in the 2.3.5 kernel, which I can't change because of the Droid's encrypted boot loader. Thanks, Motorola.

On reflection, though, I realized this was actually a great way to make sure my code was robust - both the client and server needed to detect when the wireless connection dropped out and recover, gracefully and quickly. That's working fairly well now - both ends usually detect it and they reconnect after a few seconds. This would be a real problem for an aerial or underwater vehicle, but a ground rover can just stop and wait for the connection to come back up. Either way, it's excellent practice for coding on the unreliable internet.

Once the code is done, I'll post it for anyone looking to do something similar, but it's not as reliable as I would like at the moment.

Remotely operated rover - Andoid and IOIO Basics

When I first started the project, I had never written an Android program before. I've built simple Atmel-AVR and Basic Stamp robots, and usually the first thing you do is make sure you can blink an LED from the microprocessor - it's "Hello, world" for micros. First, I needed to get the compiler up and running for Android, and understand something about how Android works.

I started with the excellent Android dev tutorials and worked through the first few programs. I then followed the tutorial at Sparkfun to get the IOIO sample projects working, and tinkered with them a bit. I found that the IOIO worked great on my Galaxy Nexus and Kyros 7127. My older Droid X2 needs the USB connection set to "charge only" to work. I plan on using the Droid for the rover. I am pretty impressed with the design of the development kit for Android, and extremely impressed with the IOIO. A great deal of thought has gone into making it easy to program for and interface things with.

 I first experimented with the Hello IOIO application, which runs a GUI application and a thread to talk to the IOIO. It worked fine, and is a great way to test the IOIO on your phone. The Hello application let's you turn an LED on the IOIO on and off from the phone.

 The sample programs also include a sample service, that runs in the background and only communicates with the user via statuses. I decided this was the way to go for controlling a rover. Android apps have life cycle that you must track to deal with incoming calls, screen rotation, and other events. That actually tears down the program and restarts it, and your program has to deal with it. This is not ideal for a realtime control application.

The service makes it a little trickier to provide feedback and debug, but you can use the logging feature to get what you need. A robot or rover will be talking to the user over the network anyway, right? :-)

 The next step was to decide how to handle the networking aspects, which proved to be very interesting. That will be the topic of the next post.

Remotely operated rover with Android and IOIO - Goals

Hello, interested reader. I have benefited so much from other people's blog posts, both in my hobbies and my professional life, that I decided it was high time I contributed a little. I hope that someone finds it useful.

I've been interested in remotely operated vehicles for some time -  projects like those on DIY Drones, OpenROV, and plenty of others are absolutely fascinating to me. I like the idea of being able to send a machine into inaccessible areas for science or exploration. I decided to focus on a network-driven rover at first, with the intent of learning what it takes to drive something over a network.

My intend is to document that process here. I've seen similar projects on the web, but I've never seen the design process documented - the tradeoffs, the software architechture, the naive assumptions that proved wrong. So that's what I'll try to do.

The idea was to send commands from a computer or tablet to an Android phone, which would interface to the very cool IOIO board. Android phones have the advantage of packing a lot of processing power into a small, low power package, along with numerous sensors. That makes it a pretty attractive platform for a remotely operated vehicle (ROV) or robot.

I wanted to be able to send back sensor data from the rover, such as video, voltage levels, accelerometers, GPS data, etc. and display them on a simple console. So on the network, the command traffic would look like:

rover sends current sensor data to console
console sends back commands (turn on motors, etc)

Video would be handled on a separate connection.

This seems simple. If you've ever done socket and thread programming, it probably is. If you've never done that, or written a program for Android, it's not so simple, and it's an excellent learning experience. It's also a ton of fun.. getting an LED to turn on in response to a command over the network the first time is seriously cool.

With that basic overview, future posts will detail what I'm learning as I go, what works, what didn't, etc.