Sunday, February 17, 2013

WIFI Signal Strength in Android

This evening I did some significant cleanup on the Android code that will control the robot/rover. I also decided it would be nice to be able to tell if you were about to drive off past your WIFI range, so I looked at how it can be measured.

 I had added a WifiManager object earlier to lock the Wifi so that it would not be throttled back by the phone. It quite reasonably does this to save power, but you don't want that to happen if you are using the wireless link to drive around, so the application requests a lock at startup and releases it on shutdown.

 If you have a WifiManager, it's easy to ask it the current state of the link, and it will return a bunch of information including signal strength in dB. If you want it to report "bars", it has a function to compute how many bars you are getting on whatever scale you prefer. Since I want a simple color coded indicator on the client control panel, I just went for a 0-5 bar scale.

 Here's the relevant code for reading signal strength and requesting/releasing WIFI locks:

import android.net.wifi.WifiManager;
import android.net.wifi.WifiManager.WifiLock;
import android.net.wifi.WifiInfo;

//owned by the class

WifiManager wifiManager = null;
WifiLock lock = null;
Integer signalStrength = 0;

//Prevent Android from throttling the wifi back to save batteries
private void obtainWifiLock()
{
if (!lock.isHeld()) 
 lock.acquire(); 
}
 
private void releaseWifiLock()
{
if (lock != null) 
     if (lock.isHeld()) 
            lock.release(); 
}
 
 
private void updateWifiStats()
{
//currently just updates the WIFI signal level
Integer numLevels = 6;
  
WifiInfo currentInfo = null;
 
currentInfo = wifiManager.getConnectionInfo();
signalStrength = wifiManager.calculateSignalLevel(currentInfo.getRssi(), numLevels);
  
}

Dirt simple video streaming

As mentioned in the last post, I found that even a very crude Java client could pull images from the Android phone's IP Webcam application at 320x200 fast enough for acceptable video. Good video streaming is a whole complex subject in itself, but I was surprised that this produced functional results. Here's standalone test code that's about as simple a streaming viewer as you are likely to find.

A simple launcher class:
import java.awt.*;
import javax.swing.*;
import java.io.*;
import java.awt.event.*; 
import javax.imageio.*;
import java.awt.image.*;

import javax.swing.event.*;

import java.net.*;
import java.io.*;

public class launcher
{


public static void main(String[] args)
{
int count = 0;

MJpegViewer b = new MJpegViewer();

JFrame frame2 = new JFrame();
frame2.setSize(600,450);
frame2.setTitle("Video Feed");
frame2.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
frame2.add(b);
 
//center frame on screen
//frame2.setLocationRelativeTo(null);
 
//set absolute screen position
frame2.setLocation(250, 0);
frame2.setVisible(true);

Image image = null;

try {
    URL url = new URL("http://192.168.2.104:8080/shot.jpg"); 
      
 

while (count < 10000)
 {  
    image = ImageIO.read(url);
  
 BufferedImage buffered = (BufferedImage) image;
 b.setBufferedImage(buffered);
 count = count + 1;
 }

} catch (IOException e) {}

}

}

The extended image frame:
import java.awt.*;
import javax.swing.JComponent;
import java.util.Scanner;
import javax.swing.*;
import java.awt.*;
import java.io.*;
import java.awt.event.*;
import java.awt.geom.*;
import java.awt.image.*;
import javax.imageio.*;
import java.lang.Math;

public class MJpegViewer extends JComponent {

private static BufferedImage img = null;

public void setBufferedImage(BufferedImage newImg)
{
img = newImg;
repaint();
}



public void paintComponent(Graphics g) {
           
  //System.out.println("In paintComponent"); 
  g.drawImage(img, 0, 0, 600, 450, null);
      }


}

Saturday, February 9, 2013

Streaming Video From the Rover

While experimenting with the excellent IP Webcam video server for Android, I found that the standard web browser and programs like VLC were intolerant of disconnects of the WiFi. Each time it disconnected, it would require manual intervention to reconnect, so I started looking at what it would take to make a very simple viewer in my Java client program.

It turns out to be relatively easy, though there is plenty of room for improvement. I found that even on my little netbook, I could get smooth video at 320x200 from the Android server just by repeatedly pulling down a static JPG from the IP Webcam and shoving each image into a simple extended JContainer. I expected it to be jerky, but it worked surprisingly well.

Note: I first tried a JLabel, which is the simplest way to load a picture in Java that I'm aware of. It was VERY slow. Not recommended. :-)

While it would likely be a bit faster to read the MJPEG stream it can provide, this was much easier to get working, and should be fine for what I want to do with it.

Once I had it working in a test program, I added a dedicated thread to my client program to handle the video connection and display. So now the client program consists of a GUI thread, a data/command thread, and a video thread.

The video thread launches and goes into a nested loop. The outer loop is keyed to a flag that gets set when the user requests a connection to the robot. The inner loop grabs the images from the Android server. If the connection drops or an image retrieval error occurs, it breaks out of the inner loop, resets and continues retrying until the user requests a disconnect. As a result, like the command/sensor thread, it automatically reconnects after the WiFi connects back up, and resumes the video feed.

The handling of the loss of the network connection has been consistently the hardest part of this so far, but that nested loop approach has worked well in both threads to automatically re-establish the connection.


Communications and Control - Initial Planning

I've built some simple robots, mostly of the "drive around and use IR or ultrasound to not run into things" variety. Once I had a basic back-and-forth network socket program working, it was time to think about overall program flow.

A traditional robotics paradigm is the "Sense, Think, Act" cycle. A robot takes input from it's sensors, performs processing on them to try to identify the best course of action, and then commands the robot's actuators to do something. The process then repeats.


At the moment, I'm not building a robot in the typical sense. That's because a human is in the loop, making the decisions based on sensor input. I wanted to make sure that the platform could be used as a robot, just by changing the software on the phone, but right now I'm interested in building a reasonably robust remotely operated vehicle. I'll continue to use "robot" because it's convenient. :-)

On reflection, I decided that a remotely operated vehicle can follow the same sense-think-act cycle. The primary difference is that the thinking is done off-vehicle, by the human operator.


With that understanding, I started thinking about how the communications will work. I intend to use the excellent IPCam program to stream the video from the phone. It works great, is robust, can run as a service, and can auto-start when the phone boots up. 

The rest of the program will run in a program based on the IOIO service example, described in a previous post. Thus, the video stream will be separate from the command and sensor stream.


I've run a test with the IOIO service and IPCam, and found that streaming video and sending sensor/command data back and forth at the same time works fine. I just used a browser for the video stream, and my little Java program for the sensor/command data. 

I did find that neither the browser or VLC will attempt to reconnect to the video server on the phone if the connection drops. I may decide to integrate a simple MJPEG viewer into the Java client to make it reconnect automatically, as the command/sensor connection does. Doing so would also be a good step towards allowing control from a phone or tablet, rather than a laptop.