NXT Robot – Smartlab

The goal of this project is to connect an old Android smartphone (e.g. HTC Magic) to a Lego NXT robot, in order to get a global available WIFI GPRS UMTS camera robot.

The easiest way to monitor your household from anywhere in the world is to connect an old Android smartphone with a mobile NXT robot. A NXT robot represents a cheap and simple to handle microcontroller device in combination with two servos, that allow to build a flexible mobile platform. On top of this mobile robot we place the Android smartphone. The Android smartphone acts as high-level controller that receives commands over the Internet and transmits these commands to the NXT controller over a Bluetooth connection.

To offer the highest flexibility of control, we implement a tiny HTTP server (by using nanohttpd library) that runs as an Android background service. This HTTP server waits for incoming requests and serves a single HTML5 robot control Web page. This page shows the actual camera image, taken from the Android device, as well as basic control buttons for moving the NXT robot around:

<!DOCTYPE html>
<html>
<head>
<meta charset="ISO-8859-1">
<title>Smartlab Cam</title>
<script type="text/javascript" src="jquery.js"></script>       
<script type="text/javascript">
	$(document).ready(function() {
	   $("#left").mousedown(function() {
	     $.get('left');
	   });
	   $("#left").mouseup(function() {
		     $.get('stop');
	   });
	   $("#up").mousedown(function() {
		  $.get('up');
	   });
	   $("#up").mouseup(function() {
		     $.get('stop');
	   });
	   $("#down").mousedown(function() {
		  $.get('down');
	   });
	   $("#down").mouseup(function() {
		     $.get('stop');
	   });
	   $("#right").mousedown(function() {
		  $.get('right');
	   });
	   $("#right").mouseup(function() {
		     $.get('stop');
	   });
	});

	var pwd = "";

	function start() {	
		pwd = prompt("Password");
		setTimeout("update()", 1000);
		var drawingCanvas =

document.getElementById('canvas');

	
		drawingCanvas.width  = window.innerWidth;
		drawingCanvas.height = window.innerHeight / 2;
	}

	function update () {
		var drawingCanvas =

 document.getElementById('canvas');

		// Check the element is in the DOM

 // and the browser supports canvas
		if(drawingCanvas.getContext) {
			// Initaliase a 2-dimensional drawing context
			var context =

                           drawingCanvas.getContext('2d');
			//Canvas commands go here

			var cam = new Image();
			cam.onload = function() {
			   context.drawImage(cam, 0, 0,

                                       cam.width, cam.height);
			}
			cam.src = 'cam.jpg?pwd=' + pwd;
			setTimeout("update()", 500);
		}

	}
</script>

</head>
<body onload="start()">
	<center>
	<table>
	<tr>
		<td>
		<a href="http://www.smartlab.at">
			<img src="header.jpg"/>
		</a>
		</td>
		<td>
			<img width="100" src="nxt_s.png"/>
		</td>
	</tr>
	</table>
	<div>
	<canvas id="canvas">
		<p>Your browser doesn't support canvas.</p>
	</canvas>
	</div>
	<table>
	<tr>
		<td>
			<img id='left' src="left.png"/>
		</td>
		<td>
			<img id='up' src="up.png"/>
		</td>
		<td>
			<img id='down' src="down.png"/>
		</td>
		<td>
			<img id='right' src="right.png"/>
		</td>
	</tr>
	</table>
	</center>
</body>
</html>

By opening a Bluetooth connection between an Android smartphone and a NXT brick, it is possible to directly send NXT commands from an app to a NXT robot.

For details on the Lego NXT protocol specification and direct command specification please study following Lego documents: Appendix 1-LEGO MINDSTORMS NXT Communication protocol and LEGO_MINDSTORMS_NXT_Direct_commands.

package at.smartlab.lego;

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;

public class NxtBrick {

	private OutputStream out;
	private InputStream in;

	/**
 * The constructor needs both io streams 
 * to communicate with a NXT brick.
 *
 * @param out the stream to send commands to
 * @param in the stream to read responses from
 */
	public NxtBrick(OutputStream out, 
			InputStream in) {
		this.out = out;
		this.in = in;
	}

	/**
 * Play a tone on the Nxt brick.
 *
 * @param frequency 
 * @param msec duration 
 */
	public void playTone(char f, char msec) 
			throws IOException {
		byte [] msg = new byte [6];
		msg[0] = (byte)0x80;
		msg[1] = (byte)0x03;
		msg[2] = (byte)(f & 0xff);
		msg[3] = (byte)((f >> 8) & 0xff);
		msg[4] = (byte)(msec & 0xff);
		msg[5] = (byte)((msec >> 8) & 0xff);
		sendMessage(msg);
	}

	public void setOutputState(byte motor, 
			byte power, 
			boolean speedReg, 
			boolean motorSync, 
			byte runState) throws IOException {
		byte[] msg = { (byte)0x80, 0x04, motor, power, 
				0x01, 0x01, 0x33, runState, 0x00, 
				0x00, 0x00, 0x00 };

		sendMessage(msg);
	}

	private void sendMessage(byte [] msg) throws IOException {
		if(out !=null) {
			out.write(msg.length & 0xff);
			out.write((msg.length >> 8) & 0xff);
			out.write(msg);
			out.flush();
		}
	}
}

The relevant Android app part that is responsible for Bluetooth device discovery as well as to open a serial connection stream to a NXT brick looks as follows:

// Create a BroadcastReceiver for ACTION_FOUND
private final BroadcastReceiver 
  mReceiver = new BroadcastReceiver() {

  @Override
  public void onReceive(Context context, 
			Intent intent) {
    String action = intent.getAction();
    // When discovery finds a device
    if (BluetoothDevice.ACTION_FOUND.equals(action)) {
      // Get the BluetoothDevice object from the Intent
      BluetoothDevice device = 
        intent.getParcelableExtra(BluetoothDevice.EXTRA_DEVICE);
      BluetoothClass btc = device.getBluetoothClass();

      if(device.getName().equals("NXT")) {
        Log.d("NxtBotGuard", "Ok device found:" + device.getName());
	Method m;
	try {
	  m = device.getClass().getMethod(
            "createRfcommSocket", new Class[] {int.class});
	  final BluetoothSocket socket = 
            (BluetoothSocket) m.invoke(device, 1);
	  socket.connect();
	  NxtBrick nxt = new NxtBrick(socket.getOutputStream(), 
            socket.getInputStream()));
	} catch (Exception e) {
	  e.printStackTrace();
	}
      }
    }
  }
};

I designed and built a quite simple mobile Lego robot that holds plenty of space on its back in order to carry a long lasting 7.2 V battery. This battery provides the power for the NXT robot as well as for the HTC magic smartphone (via mini USB plug and a 5V voltage converter).

Following HTML5 page shows the basic controller interface the NXT robot is serving directly through its nanohttpd HTTP server:

NXTBotGuard Source

Leave a Reply