Category: Interaction Design

How to install Scratch for Arduino (S4A) on Raspberry Pi

Scratch for Arduino is an extension on Scratch for controlling a connected Arduino. It is a great way to start programming and building electronics.

  • First thing to do is start your Raspberry Pi, and make sure it is connected to the internet.
  • Open the browser (Midori) and go to www.s4a.cat.
  • On that page go to Downloads, click on the link that says “Raspbian (Debian for RaspberryPi)” and choose “Save”. This will save the file “S4A15_RP.deb” to /home/pi/.
  • Now open a terminal window, by clicking on LXTerminal on the desktop. In this window you can type commands to make your computer do things. Now we want to tell it to install the file we just downloaded.
  • The location you are at when you open the terminal should be /home/pi/. You can check this by typing the command pwd and pressing enter. It will then show you in which directory you are working.
  • This should also be the place we saved the file. We can check this by typing ls and enter. It will then show you all files and directories inside the directory we are now. In this list there should be a file called “S4A15_RP.deb” or similar.
  • Now let’s install it. For this we will use the command dpkg. But to run this command the computer demands that we have the rights to install this. Therefore we need to add the command sudo to execute the command as a super user. So in the end, the command you need to type becomes: sudo dpkg -i S4A15_RP.deb . It is important to write it with capital letters where needed, because otherwise the computer does not know what file you are talking about.
  • The installation will take a little while, and the computer will show you what it is doing by writing all kinds of text. When it is done (you see pi@raspberrypi ~$ again) you can close the window.

Commands in the terminal

  • Now if you open the start menu and go to Programming, there should be S4A in the list. Click on it to start Scratch for Arduino.
  • Of course we also need an Arduino to program on. This Arduino needs to be prepared for S4A by loading a specific ‘sketch’ (an Arduino code file) onto it. This file can also be found on www.s4a.cat under Downloads (heading ‘Firmware’).
  • If you have access to a computer with the Arduino software already installed, it is easiest to do it from there. Open and upload the file to your Arduino as a regular Arduino code.
  • Note that not all Arduino’s work well with Scratch for Arduino. Currently only the Arduino Diecimila, Duemilanove and Uno are tested.
  • When you start S4A you will see a drawing of an Arduino already loaded as a sprite.
  • It will automatically search for the Arduino. If this does not find it, right click on the sensor board (the list with all the blue values) and select “stop searching board”.
  • Right click it again and click “Select Serial/USB-port”. Then there should pop up a little menu where you can select the port of your connected Arduino.

Steer a Double Robotics robot with your Android’s accelerometer

Double Robotics hack from Niels Swinkels on Vimeo.

Hjulius
The office where I work recently bought a Double Robotics telepresence robot. Imagine an iPad ontop of a Segway, and you will have a pretty good idea of how it looks and behaves. The fun thing of course is that you can start a video call to it. Your face will show up on the iPad, and you can see what the robot sees and tell it where to go.

One thing that disappointed me a bit though was that only iPhones, iPads, or computers with the Chrome browser could be used to connect to the robot. I mean, why can I not do it from the Chrome browser on my Android phone? After a short dive into the source code of the Double Robotics webpage I figured out a way to make it possible, with a little bonus functionality in the end.

Let’s dive in

Everything happens on the page http://drive.doublerobotics.com. First you log in. Then you connect to your robot, and start the video call. The interface changes, you see what the robot sees, and you can steer it. After the logging in most is done with the help of Javascript. This is quite easy to figure out by reading the source code of the webpage. On line 31 for example you see a reference to a Javascript file called drive.js. There you can find all the functions used to control the robot. The fun thing with Javascript is that we, as the visitor of the webpage, can use it as well.

Make a connection

If you have not done so already, go to http://drive.doublerobotics.com and log in with your account. Right-click somewhere on the page, and select View page source. If you do not see it in the menu, try right-clicking on a piece of text on the page. Or via the menu: Menu > Tools > View source (Ctrl+U) (Chrome).
Now search (Ctrl+F) for connectTo. You will find something like this:
connectTo('CC5DC445-9455-4D69-8ED3-95EA7D1BB1D6');
Copy this to somewhere (notepad) so you can use it later. This function is called to start the video call with your robot. I assume that the number is some form of serial number. If you now type the following in your address bar, you should see the connection starting (Use the serial number you just found):
javascript:connectTo('CC5DC445-9455-4D69-8ED3-95EA7D1BB1D6');

One small step..

As I wrote before, in the file drive.js you can find the functions used to drive the robot. Driving and steering is done with one function:
sendCommandWithData(8, { "drive" : -100, "turn" : 0 });
Here the drive number means how far you go forward (negative numbers) or backwards (positive numbers), and the turn means rotate right (negative numbers) or left (positive numbers). The default speed for going backwards is 50, and for making a turn is 100 or -100. Feel free to experiment with other numbers, but make sure you catch your robot if it falls! It seems that the numbers represent the amount of movement, and not the speed with which you move.

What more can we do?

Here are some more functions you can call. This is not a complete list. If you want to find all possible actions, try searching in the Javascript yourself.

sendCommandWithData(9, { "pole" : 200 });
Where 200 means move the pole up, -200 means down. Smaller numbers mean less movement. In this way you can position the pole at any height you want, not just all the way up or down. The only weird thing is that the robot might think that the pole has moved all the way up or down, and refuses to move more. To go around this you just move it once in the opposite direction, and then quickly a few times in the direction you want it to go.
sendCommand(10);
Park
sendCommand(11);
Un-park
sendCommand(26);
Switch camera

I’m lazy.. less typing please?

So all of these functions you can call by typing javascript: followed by your list of commands in the address bar. I admit, that is not the easiest way to drive a robot. A better way is to create bookmarks. (In Chrome:) Click menu > Bookmarks > and put a checkmark in front of Show bookmarks bar. Then create a new bookmark by right-clicking the bookmarks bar and selecting Add page.... As name write Forward and as url javascript:sendCommandWithData(8, { "drive" : -100, "turn" : 0 });. Now you can click your Forward bookmark to make the robot drive one step forward. In the same way you can create more buttons for the other functions.

forward bookmark

Go mobile

If you felt adventurous and tried this in the Chrome browser of an Android phone, you will have noticed that opening a bookmark via the menu does not work in this case. That is because as soon as you open the bookmark menu, you leave the Double Robotics webpage. The way around that problem is to start typing the name of your bookmark in the address bar, and click on it when you see it as a suggestion.
forward bookmark mobile

Sync bookmarks

For those of you who have set up all the bookmarks in Chrome on their computer, and now want to have them on their mobile Chrome as well, there is good news. Our ‘good old friend’ Google has built in a synchronization functionality, which copies your bookmarks between devices. To turn it on do the following:

  1. In Chrome on the computer, go to Menu > Sign in to Chrome... and sign in
  2. Then go to Menu > Settings and click on Advanced sync settings...
  3. There you can select what should be synced between devices. In our case we only need bookmarks.
  4. On your Android phone, open Android settings, and find Google under the heading Accounts
  5. Click on your account, and select Chrome
  6. Under the menu (The three dots in the upper right corner) select sync now
  7. You should now find your bookmarks in your mobile Chrome in a folder called Desktop bookmarks

Droids in control

And at this point we can now connect to our robot, from an Android phone, even though Double Robotics tells us we can not. Just log in, and use the connectTo function.

More customization!

Still this whole business of bookmarks is not really that awesome, specially not on a small touchscreen. Good thing that Javascript has some more tricks up its sleeve. What if I told you we can add our own interface into the webpage? Add this next code as one bookmark, and use it after having started a connection to your robot:

javascript:var interface = document.getElementById('sessionButtonsColumn');var btnForward = document.createElement('button');btnForward.setAttribute('onclick', 'sendCommandWithData(8, { "drive" : -100, "turn" : 0 });');btnForward.innerHTML='forward';interface.appendChild(btnForward);var btnBackward = document.createElement('button');btnBackward.setAttribute('onclick', 'sendCommandWithData(8, { "drive" : 50, "turn" : 0 });');btnBackward.innerHTML='backward';interface.appendChild(btnBackward);var btnLeft = document.createElement('button');btnLeft.setAttribute('onclick', 'sendCommandWithData(8, { "drive" : 0, "turn" : 100 });');btnLeft.innerHTML='left';interface.appendChild(btnLeft);var btnRight = document.createElement('button');btnRight.setAttribute('onclick', 'sendCommandWithData(8, { "drive" : 0, "turn" : -100 });');btnRight.innerHTML='right';interface.appendChild(btnRight);

This will add four buttons to your interface which you can click to drive the robot forward, backward, left or right. It works by finding the part of the interface which contains the already existing buttons
document.getElementById('sessionButtonsColumn')
and appending four new buttons to it.
interface.appendChild(btnForward);
interface.appendChild(btnBackward);
interface.appendChild(btnLeft);
interface.appendChild(btnRight);

With a little bit of Html knowledge and some experimentation you can change this code to create any shape of interface you want.

Are we done?

Nope. Because one of the latest cool things on the web has been Html 5. And one of the things that makes it cool, is that it gives you access to the accelerometer of your smartphone. Html 5 is for a big part Javascript, so it is a piece of cake to add this coolness to our robot script.

Based on this very clean example of how to read the sensor values (Thanks Piotr and Oskar!), I made this code:


javascript:
if (window.DeviceMotionEvent) {
window.addEventListener('devicemotion', function(ev) {
var acc = ev.accelerationIncludingGravity;
dmHdlr(acc.x, acc.y, acc.z);
}, false);
}
else {
alert("devicemotion not supported on your device or browser.");
}

var lastDM = new Date().getTime();
function dmHdlr(aX, aY, aZ) {
var currDM = new Date().getTime();
if (currDM < lastDM + 500) {return;} lastDM = currDM; var drive = 0; var turn = 0; if(aX < -3 || aX > 3)
{
turn = aX*10;
}
if(aY < -3 || aY > 3)
{
drive = aZ * -10;
}
sendCommandWithData(8, { "drive" : drive, "turn" : turn });
}

Add this, as one long line, as bookmark to your browser. Then connect to your robot, and select the bookmark. Now you can steer your robot by just tilting your phone! Hold it upright to stand still. Tilt the top away from you to go forward, or towards you to go backwards. Tilt the screen to the left to turn left, and to the right to turn right. Enjoy! Note: Please drive carefully with your robot, and do not forget you have quite a delay in response time.

Custom touch controls

Carmageddon interface
The green boxes indicate that these controls can be moved and how big their touch area is.

Today the old ‘racing’ game Carmegeddon was released as a port to Android. When it first came out in 1997 it caused worldwide controversy due to that the player is encouraged to drive over pedestrians.

Today’s free tryout of the app allowed for some nostalgic pixelated blood spatting. But it was what I found in the settings which caused me to write this post. The car is driven via five onscreen buttons (left, right, gas, brake, handbrake). Via the menu this interface can be customized by dragging around the buttons to any place that suits your needs. The nice thing is that with this it can be made to fit different sizes of hands and screens. I guess this is the touch screen equivalent of changing which keyboard keys you want to use to play the game.

The free version of the app

Buddy Cup

The 'technology' behind Budweiser's Buddy Cup
The ‘technology’ behind Budweiser’s Buddy Cup (From digitaltrends.com)

Budweiser has introduced a new concept that they will use during events: the Buddy Cup. It is a beer cup that you link to your Facebook account via the QR-code on the bottom when you enter the bar and receive the cup. Then during the event, when you toast with somebody (and your cups touch), you automatically become friends on Facebook.

The initial idea sounds fun. It is an existing meaningful gesture which now has an additional digital meaning. But have they really thought this through? What if you bump into someone by accident? Or what if a random person walks by and bumps your cup without your approval, that person can then instantly see all private information you have shared with your friends. And this means you need to keep your cup with you all evening long, instead of getting a new one with each drink as is normally the case (or is this different in America?).

To me this idea sounds like nothing more than initial idea that needs a lot more thought before it can become a real product.

Funny detail: In the movie they use the form of the Facebook time line to show the ‘history of toasting’. But their time line goes in the opposite direction as the one from Facebook (scroll down = back in time).

www.youtube.com/watch?v=9FdnssHlcRk