A traditional robotics paradigm is the "Sense, Think, Act" cycle. A robot takes input from it's sensors, performs processing on them to try to identify the best course of action, and then commands the robot's actuators to do something. The process then repeats.
At the moment, I'm not building a robot in the typical sense. That's because a human is in the loop, making the decisions based on sensor input. I wanted to make sure that the platform could be used as a robot, just by changing the software on the phone, but right now I'm interested in building a reasonably robust remotely operated vehicle. I'll continue to use "robot" because it's convenient. :-)
On reflection, I decided that a remotely operated vehicle can follow the same sense-think-act cycle. The primary difference is that the thinking is done off-vehicle, by the human operator.
With that understanding, I started thinking about how the communications will work. I intend to use the excellent IPCam program to stream the video from the phone. It works great, is robust, can run as a service, and can auto-start when the phone boots up.
The rest of the program will run in a program based on the IOIO service example, described in a previous post. Thus, the video stream will be separate from the command and sensor stream.
I've run a test with the IOIO service and IPCam, and found that streaming video and sending sensor/command data back and forth at the same time works fine. I just used a browser for the video stream, and my little Java program for the sensor/command data.
I did find that neither the browser or VLC will attempt to reconnect to the video server on the phone if the connection drops. I may decide to integrate a simple MJPEG viewer into the Java client to make it reconnect automatically, as the command/sensor connection does. Doing so would also be a good step towards allowing control from a phone or tablet, rather than a laptop.