Tag: android

Double Robotics hack from Niels Swinkels on Vimeo.

The office where I work recently bought a Double Robotics telepresence robot. Imagine an iPad ontop of a Segway, and you will have a pretty good idea of how it looks and behaves. The fun thing of course is that you can start a video call to it. Your face will show up on the iPad, and you can see what the robot sees and tell it where to go.

One thing that disappointed me a bit though was that only iPhones, iPads, or computers with the Chrome browser could be used to connect to the robot. I mean, why can I not do it from the Chrome browser on my Android phone? After a short dive into the source code of the Double Robotics webpage I figured out a way to make it possible, with a little bonus functionality in the end.

Let’s dive in

Everything happens on the page http://drive.doublerobotics.com. First you log in. Then you connect to your robot, and start the video call. The interface changes, you see what the robot sees, and you can steer it. After the logging in most is done with the help of Javascript. This is quite easy to figure out by reading the source code of the webpage. On line 31 for example you see a reference to a Javascript file called drive.js. There you can find all the functions used to control the robot. The fun thing with Javascript is that we, as the visitor of the webpage, can use it as well.

Make a connection

If you have not done so already, go to http://drive.doublerobotics.com and log in with your account. Right-click somewhere on the page, and select View page source. If you do not see it in the menu, try right-clicking on a piece of text on the page. Or via the menu: Menu > Tools > View source (Ctrl+U) (Chrome).
Now search (Ctrl+F) for connectTo. You will find something like this:
connectTo('CC5DC445-9455-4D69-8ED3-95EA7D1BB1D6');
Copy this to somewhere (notepad) so you can use it later. This function is called to start the video call with your robot. I assume that the number is some form of serial number. If you now type the following in your address bar, you should see the connection starting (Use the serial number you just found):
javascript:connectTo('CC5DC445-9455-4D69-8ED3-95EA7D1BB1D6');

One small step..

As I wrote before, in the file drive.js you can find the functions used to drive the robot. Driving and steering is done with one function:
sendCommandWithData(8, { "drive" : -100, "turn" : 0 });
Here the drive number means how far you go forward (negative numbers) or backwards (positive numbers), and the turn means rotate right (negative numbers) or left (positive numbers). The default speed for going backwards is 50, and for making a turn is 100 or -100. Feel free to experiment with other numbers, but make sure you catch your robot if it falls! It seems that the numbers represent the amount of movement, and not the speed with which you move.

What more can we do?

Here are some more functions you can call. This is not a complete list. If you want to find all possible actions, try searching in the Javascript yourself.

sendCommandWithData(9, { "pole" : 200 });

Where 200 means move the pole up, -200 means down. Smaller numbers mean less movement. In this way you can position the pole at any height you want, not just all the way up or down. The only weird thing is that the robot might think that the pole has moved all the way up or down, and refuses to move more. To go around this you just move it once in the opposite direction, and then quickly a few times in the direction you want it to go.

sendCommand(10);

Park

sendCommand(11);

Un-park

sendCommand(26);

Switch camera

I’m lazy.. less typing please?

So all of these functions you can call by typing javascript: followed by your list of commands in the address bar. I admit, that is not the easiest way to drive a robot. A better way is to create bookmarks. (In Chrome:) Click menu > Bookmarks > and put a checkmark in front of Show bookmarks bar. Then create a new bookmark by right-clicking the bookmarks bar and selecting Add page.... As name write Forward and as url javascript:sendCommandWithData(8, { "drive" : -100, "turn" : 0 });. Now you can click your Forward bookmark to make the robot drive one step forward. In the same way you can create more buttons for the other functions.

Go mobile

If you felt adventurous and tried this in the Chrome browser of an Android phone, you will have noticed that opening a bookmark via the menu does not work in this case. That is because as soon as you open the bookmark menu, you leave the Double Robotics webpage. The way around that problem is to start typing the name of your bookmark in the address bar, and click on it when you see it as a suggestion.

Sync bookmarks

For those of you who have set up all the bookmarks in Chrome on their computer, and now want to have them on their mobile Chrome as well, there is good news. Our ‘good old friend’ Google has built in a synchronization functionality, which copies your bookmarks between devices. To turn it on do the following:

1. In Chrome on the computer, go to Menu > Sign in to Chrome... and sign in
2. Then go to Menu > Settings and click on Advanced sync settings...
3. There you can select what should be synced between devices. In our case we only need bookmarks.
4. On your Android phone, open Android settings, and find Google under the heading Accounts
5. Click on your account, and select Chrome
6. Under the menu (The three dots in the upper right corner) select sync now
7. You should now find your bookmarks in your mobile Chrome in a folder called Desktop bookmarks

Droids in control

And at this point we can now connect to our robot, from an Android phone, even though Double Robotics tells us we can not. Just log in, and use the connectTo function.

More customization!

Still this whole business of bookmarks is not really that awesome, specially not on a small touchscreen. Good thing that Javascript has some more tricks up its sleeve. What if I told you we can add our own interface into the webpage? Add this next code as one bookmark, and use it after having started a connection to your robot:

javascript:var interface = document.getElementById('sessionButtonsColumn');var btnForward = document.createElement('button');btnForward.setAttribute('onclick', 'sendCommandWithData(8, { "drive" : -100, "turn" : 0 });');btnForward.innerHTML='forward';interface.appendChild(btnForward);var btnBackward = document.createElement('button');btnBackward.setAttribute('onclick', 'sendCommandWithData(8, { "drive" : 50, "turn" : 0 });');btnBackward.innerHTML='backward';interface.appendChild(btnBackward);var btnLeft = document.createElement('button');btnLeft.setAttribute('onclick', 'sendCommandWithData(8, { "drive" : 0, "turn" : 100 });');btnLeft.innerHTML='left';interface.appendChild(btnLeft);var btnRight = document.createElement('button');btnRight.setAttribute('onclick', 'sendCommandWithData(8, { "drive" : 0, "turn" : -100 });');btnRight.innerHTML='right';interface.appendChild(btnRight);

This will add four buttons to your interface which you can click to drive the robot forward, backward, left or right. It works by finding the part of the interface which contains the already existing buttons
document.getElementById('sessionButtonsColumn')
and appending four new buttons to it.
interface.appendChild(btnForward);
interface.appendChild(btnBackward);
interface.appendChild(btnLeft);
interface.appendChild(btnRight);
With a little bit of Html knowledge and some experimentation you can change this code to create any shape of interface you want.

Are we done?

Nope. Because one of the latest cool things on the web has been Html 5. And one of the things that makes it cool, is that it gives you access to the accelerometer of your smartphone. Html 5 is for a big part Javascript, so it is a piece of cake to add this coolness to our robot script.

Based on this very clean example of how to read the sensor values (Thanks Piotr and Oskar!), I made this code:


javascript:
if (window.DeviceMotionEvent) {
window.addEventListener('devicemotion', function(ev) {
var acc = ev.accelerationIncludingGravity;
dmHdlr(acc.x, acc.y, acc.z);
}, false);
}
else {
alert("devicemotion not supported on your device or browser.");
}

var lastDM = new Date().getTime();
function dmHdlr(aX, aY, aZ) {
var currDM = new Date().getTime();
if (currDM < lastDM + 500) {return;} lastDM = currDM; var drive = 0; var turn = 0; if(aX < -3 || aX > 3)
{
turn = aX*10;
}
if(aY < -3 || aY > 3)
{
drive = aZ * -10;
}
sendCommandWithData(8, { "drive" : drive, "turn" : turn });
}

Add this, as one long line, as bookmark to your browser. Then connect to your robot, and select the bookmark. Now you can steer your robot by just tilting your phone! Hold it upright to stand still. Tilt the top away from you to go forward, or towards you to go backwards. Tilt the screen to the left to turn left, and to the right to turn right. Enjoy! Note: Please drive carefully with your robot, and do not forget you have quite a delay in response time.