As our Tangible Interaction Workshop is close to the end, I’m been thinking about the idea of ‘tangible and intuitive control’. In a way I wan to create a controller which requires minimal effort to learn: as you touch it, you will know how to use it in seconds. I decided to rework on ‘game controller’ and integrate what I’ve developed for the music player.

Music Player / Game Controller

Game UX

For this game I borrow the code from, a game library from p5.js, and the music payer code written by Tom Igoe , then integrate two code in JavaScript. Specifically I want to create a controller for this Astroids Game, and make it more interactive in terms of color, music, and control experience.

Screen Shot 2017-03-21 at 12.11.28 PM
Original game (no sound)


Screen Shot 2017-03-21 at 1.42.22 PM
My game interface (with sound)

I invited my friend to play the game and this was his first time. See how he interacted with the game and controller device (he barely looked down to the controller).

Controller Design

The following strategies are what I learn from this class, in order to create an intuitive and learnable device. From a physical and tangible perspective:

• Selection of buttons and design of housing is important, since that is the first thing a new user learn from this device, before any interaction with the sensors or micro-controller. The shape, position, and movement of button have strong indication of the controlling action.

• Certain kinds of feedbacks, such as a sound or vibration after clicking or pushing the button, will enhance the learnability of the device. It is a confirmation or response to users action. This part can be integrated through coding or adding hardware such as a vibrator.

• There should be some connections between motions and meanings of control. It is helpful to think through that what exactly does a user do with the device, and how do you ask them to act on. Specifically in game control, the actions vary as continuing control (maneuvering a spaceship) and discrete action (shoot, jump…). It is important to consider what buttons you want the user to hold versus to click.

UX Diagram

Circuit and Hardware


Circuit Wiring Layout

For this controller, I used Arduino MKR1000 as micro-controller since I need the ‘keyboard’ function. For buttons, I used two push buttons and one rotary encoder with additional scrubber knob.


The general idea is that the rotary encoder control the maneuvering the space ship and when push the encoder, the spaceship shoot the astroids.

Coding in JavaScript

As I mentioned that I borrowed code from Tom and and integrate the code in JavaScript. The basic idea is to load all different sound files (game theme song, sound effects ) and associate all keyboard write with the sound to enhance the interaction and control experience. Not just to make the game fun to play, the sounds clearly indicate when the game starts and what did the user act (turn, shoot, or speed up) on the device.

var song; // the sound file to be played

// the list of songs:
//var songs = [‘Alert1.wav’,’Beep1.wav’,’Beep2.wav’,’Bounce.wav’,’Explosion1.wav’];
var songs = [‘Alert1.wav’,’Beep1.wav’,’Beep2.wav’,’Bounce.wav’,’Explosion1.wav’];
var sound = [‘Beep1.wav’,’Beep2.wav’];

var songCount = songs.length; // number of songs in the music dir
var currentSong = 0; // current song number

var soundCount = sound.length; // number of songs in the music dir
var currentSound = 0; // current song number

var bullets;
var asteroids;
var ship;
var shipImage, bulletImage, particleImage;
var MARGIN = 40;
var mySprite = allSprites[i];

//var snd = new Audio(“tone.wav”); // tone

function preload() { // load the first song on preload
song = loadSound(‘music/’ + songs[currentSong]);
snd = loadSound(‘tone/Chibi_Ninja.mp3’);
left = loadSound(‘music/Beep1.wav’);
right = loadSound(‘music/Beep2.wav’);
shoot = loadSound(‘music/Explosion1.wav’);

function setup() {
createCanvas(300, 150);

function draw() {
background(30, 20, 180);
// draw the song’s name and current time in seconds:
text(songs[currentSong], 20, 50);
text(song.currentTime().toFixed(3), 20, 100);
function controlSound(input) {
switch(input) {
case 49: // start/stop, press 7 to start the game with theme song
if (snd.isPlaying()){
} else {;

case 50: // play/pause, press 2
if (song.isPlaying()){
} else {;

case 120: // fast forward, press 5
if (snd.isPlaying()){;

function getSong(songNumber) {
if (songNumber < songs.length) { // if the song number is in range
if (song.isPlaying()) {
// load a new song:
song = loadSound(‘music/’+ songs[currentSong], resumePlay);
return true;
} else { // if the song number was out of range, return false
return false;

function resumePlay() {
// if the song isn’t playing, play it
if (song.isPlaying()){
} else {;


function keyReleased() {
controlSound(keyCode); // send the ASCII number of the key

Coding in Arduino

A major component in Arduino coding is to set the rotary encoder to sense the change of rotation, as a way to maneuver the space ship.

#include <Encoder.h>
#include <Keyboard.h>
// turn off hardware interrupts. On an M0-based board, you can skip this line
// because most of the IO pins on those boards have interrupts

Encoder encoder(6, 7); // initialize the encoder on pins 6 and 7
long lastPosition = 0; // last position of the knob
int lastButtonState = HIGH; // last state of the button
int fadeState = 0; // calculated level for the LED (0-255)
boolean onOffState = false; // whether the light is on or off

int pausePin = A0;
int pauseState = 0; // variable for reading the pushbutton status

const int shufflePin = A1; // the number of the pushbutton pin
int shuffleState = 0; // variable for reading the pushbutton status

int shootPin;
void setup() {
Serial.begin(9600); // initialize serial communication
//pinMode(2, INPUT_PULLUP); // pushbutton on pin 2
pinMode(pausePin, INPUT);
pinMode(shufflePin, INPUT);

void loop() {
//start the game
int buttonState = digitalRead(pausePin); // read pushbutton
if (buttonState != lastButtonState) { // if button has changed
delay(3); // wait a few ms (debounce delay)
if (buttonState == HIGH) { // if button is pressed
onOffState = !onOffState; // change on-off state¨
lastButtonState = buttonState; // save current button state for next time
// Define how to shoot:
shootPin = digitalRead(2);
if (shootPin == HIGH) {

// this section reads the sensors and determines if they’ve changed:
long encoderPos =;
// read encoder
// calculate change in encoder:
int encoderChange = encoderPos – lastPosition;
lastPosition = encoderPos; // save current position for next time
// this section sets left and right of the spaceship:
// if (onOffState == true) { // if the game is on
if (encoderChange > 0) {
else if (encoderChange < 0) {

shuffleState = digitalRead(shufflePin);
if (shuffleState == HIGH) {
// delay(1000);
// //Serial.println(“PAUSE”);