Final Project

My project was done with Zack Dunham and Tristan Marshall

The goal of our project, feelings of history, was to show history from a new angle by using technology to create an interactive experience. Our initial brainstorming was inspired by technology like the hololens and by statistics about how little the average citizen actually knows about history.

The prototyping process took the vast majority of our time as we researched and tried to implement different ways for the arduino  to track its location and communicate wirelessly. The first method we tried for positioning was to use an accelerometer to measure the acceleration and extrapolate from that, which didn't work because as soon as a single input was lost the position would be so incorrect there was no recovery. Next we researched a few things including: a GPS chip (doesn't work because the best resolution we could achieve was about 12 meters), using the latency of the RF chips to measure distance (doesn't work because the arduino uno's CPU clock doesn't run fast enough to measure latency at a resolution less than about 12 meters), then we tried using two ultra-sonic sensors each with one half covered (we decided this was too complicated), before finally settling on just one ultra-sonic sensor and an RF transceiver to transmit the data.

Getting the RF transceivers to work was another adventure with technology, we were pretty confident that it was the only viable solution but were having a lot of difficulty getting data to actually transmit. After several hours and trying four different libraries, we realized that one of the chips we were using was a dud. After replacing it with a different one we quickly figured it out because between trying to get it to work and trying to get the chip to work we actually learned a lot about how to set up the communication.

From there it was pretty simple to hook up a water sensor, metal sensor, and a button up. The fritzing files are below. The top one is our "home base" arduino that is connected to the computer and processing, and the bottom one is our "explorer" that gets carried around and sends the data across.


We also spent some time designing a box to hold all of the components of the explorer more securely and I've included a sketch of it below.
Our arduino code is relatively simple, the explorer sends its data from the sensors whenever the button is pressed in the (x,y,z) format. The home base writes that data into the serial moniter for the processing program. The processing sketch does all of the heavy lifting, parsing the data into numbers and interpreting it into the correct sound bytes. It also contains logic to only play clips one at a time and certain clips must play before others.
Our all of our code is below.

Processing


import processing.sound.*;
import processing.serial.*;

//Creating variables
Serial myPort;  
Sound mySound;

SoundFile firstSpotMainFile;
SoundFile firstSpotExtraOneFile;
SoundFile firstSpotExtraTwoFile;

SoundFile secondSpotMainFile;
SoundFile secondSpotExtraOneFile;
SoundFile secondSpotExtraTwoFile;

PFont f; 
PImage img;

float timerStart;
float duration;

boolean firstIsActivated = false;
boolean secondIsActivated = false;

boolean firstSensOne = false;
boolean firstSensTwo = false;

boolean secondSensOne = false;
boolean secondSensTwo = false;

boolean hasStartedDrawing = false;

boolean isPlayingClip = false;

int firstAreaMin = 10;
int firstAreaMax = 30;

int secondAreaMin = 35;
int secondAreaMax = 55;

int waterSensorCheck = 50;
int metalSensorCheck = 500;

void setup() 
{
  size(1040, 663);
  //Port to recieve data from
  String portName = Serial.list()[0];
  myPort = new Serial(this, portName, 9600);
  myPort.bufferUntil('\n');

  f = createFont("Arial",16,true);
  img = loadImage("Map.png");
  firstSpotMainFile = new SoundFile(this, "Penn1.wav");
  firstSpotExtraOneFile =new SoundFile(this, "Penn2.wav");
  firstSpotExtraTwoFile = new SoundFile(this, "Penn3.wav");
  
  secondSpotMainFile = new SoundFile(this, "Virginia2.wav");
  secondSpotExtraOneFile = new SoundFile(this, "Virginia2.wav");
  secondSpotExtraTwoFile = new SoundFile(this, "Virginia3.wav");
  
  mySound = new Sound(this);
}
void draw() 
{
  //Flag to check when the code has started drawing, used so the serial event does not start checking values until drawing has started
  if(hasStartedDrawing == false)
  {
    hasStartedDrawing = true;
  }
  
  //Setting a basic background
  background(100,100,100);
  
  //Drawing the map image background
  //Map from http://www.texpertis.com/drawing-of-the-united-states/how-to-draw-a-map-of-the-usa-9-steps-with-pictures-wikihow-drawing-of-the-united-states/
  image(img, 0, 0);
  //Setting the color of the circles based on whether their experieince has been activated
  //Main Station 1
  if(!firstIsActivated)
  {
    fill(255,0,0); 
  }
  else
  {
    fill(0,255,0); 
  }
  circle(878, 225, 40);
  
  //Station 1 - extra One
  if(!firstSensOne)
  {
    fill(255,0,0); 
  }
  else
  {
    fill(0,255,0); 
  }
  circle(913, 215, 20);
  
  //Station 1 - extra One
  if(!firstSensTwo)
  {
    fill(255,0,0); 
  }
  else
  {
    fill(0,255,0); 
  }
  circle(842, 218, 20);

  //Main Station 2
  if(!secondIsActivated)
  {
    fill(255,0,0); 
  }
  else
  {
    fill(0,255,0); 
  }
  circle(898, 311, 40);
  
  //Station 2 - extra One
  if(!secondSensOne)
  {
    fill(255,0,0); 
  }
  else
  {
    fill(0,255,0); 
  }
  circle(865, 306, 20);
  
  //Station 2 - extra One
  if(!secondSensTwo)
  {
    fill(255,0,0); 
  }
  else
  {
    fill(0,255,0); 
  }
  circle(856, 328, 20);

   stroke(0,0,0);

   textFont(f,24);

   //text("Four Score...!",850,260);
   circle(mouseX, mouseY, 20);
   //println(mouseX + " " + mouseY);
}

//Base code used in all Processing files, modified from the "SerialCallResponseASCII" Arduino Tutorial
void serialEvent(Serial myPort) 
{
  //Making sure the program has started drawing before attempting to read in data
  if(hasStartedDrawing)
  {
    // read the serial buffer:
    String myString = myPort.readStringUntil('\n');
    // if you got any bytes other than the linefeed:
    myString = trim(myString);
    
    // split the string at the commas and convert the sections into integers:
    int sensors[] = int(split(myString, ','));
    
    //Making sure to only go if the proper amount of data is recieved
    if(sensors.length == 3)
    {
      //Need to have something which will detect the distance from the the home unit to the other arduino
      //May also be handled by Arduino and then gets passed in here
      //Determine how many cases we're doing to have, aiming for one main site and 2 other sensors there
      //So we have to check which site we're at, and then what the other sensors detect
      
      //Printing out the sensor values
      println(sensors[0]+ "," + sensors[1] + "," + sensors[2]);
      
      //Making sure that a clip is not currently playing when checking the data, since we do not want 2 sounds to be playing at the same time
      if(isPlayingClip == false)
      {
        //Checking if the distance sensor is getting values between the area assigned for the first area
        if(sensors[0] >= firstAreaMin && sensors[0] <= firstAreaMax)
        {
          //Making sure the first part of the experieince has been gone through before playing audio for the sensor experieinces
          if(firstIsActivated)
           {
               //If the value of the first sensor is greater than the amount we are checking against
               if(sensors[1] >= waterSensorCheck && isPlayingClip == false)
               {
                 //Making sure that the sound byte of the first area has not been played yet
                 if(firstSensOne == false)
                 {
                   //Set the flag for the sensor event being triggered
                    firstSensOne = true;
                    //Playing the sound clip
                    firstSpotExtraOneFile.play();
                    //Get the initial time for the timer
                    timerStart = millis();
                    //Setting the duration to be the length of the current clip
                    duration = firstSpotExtraOneFile.duration();
                    //Getting the duration into milliseconds
                    duration *= 1000;
                    //Set the flag for playing the clip
                    isPlayingClip = true;
                    
                 }
               }
               //If the value for the metal sensor is under the minimum value, but also not equal to zero (Zero is base value sent when not retrieving any real data from arduino)
               if(sensors[2] <= metalSensorCheck && sensors[2] > 0 && isPlayingClip == false)
               {
                 if(firstSensTwo == false)
                 {
                   //Set the flag for the sensor event being triggered
                    firstSensTwo = true;
                    //Playing the sound clip
                    firstSpotExtraTwoFile.play();
                    //Get the initial time for the timer
                    timerStart = millis();
                    //Setting the duration to be the length of the current clip
                    duration = firstSpotExtraTwoFile.duration();
                    //Getting the duration into milliseconds
                    duration *= 1000;
                    //Set the flag for playing the clip
                    isPlayingClip = true;
                 }
               } 
            }
          //Check if the first station has not yet been activated
          //Checking this after the first checks so that those sensors will not go off the first time through
          if(!firstIsActivated)
          { 
            //Set the first station to have been activated
            firstIsActivated = true;
            //Begin playing the sound clip
            firstSpotMainFile.play();
            //Get the initial time for the timer
            timerStart = millis();
            //Setting the duration to be the time of the current sound
            duration = firstSpotMainFile.duration();
            //Extending the time to be based in milliseconds
            duration *= 1000;
            //Set the flap that a clip is currently going so that timer checks can start
            isPlayingClip = true;
          } 
        }
        //If the checks for the first area do not pass, check if the distance senseor is retrieving values within the second area
        else if(sensors[0] >= secondAreaMin && sensors[0] <= secondAreaMax)
        {
          //Making sure the second part of the experieince has been gone through before playing audio for the sensor experieinces
          if(secondIsActivated)
           {
               //If the value of the second sensor is greater than the amount we are checking against
               if(sensors[1] >= waterSensorCheck && isPlayingClip == false)
               {
                 //Making sure that the sound byte of the second area has not been played yet
                 if(secondSensOne == false)
                 {
                   //Set the flag for the sensor event being triggered
                    secondSensOne = true;
                    //Playing the sound clip
                    secondSpotExtraOneFile.play();
                    //Get the initial time for the timer
                    timerStart = millis();
                    //Setting the duration to be the length of the current clip
                    duration = secondSpotExtraOneFile.duration();
                    //Getting the duration into milliseconds
                    duration *= 1000;
                    //Set the flag for playing the clip
                    isPlayingClip = true;
                 }
               }
               //If the value for the metal sensor is under the minimum value, but also not equal to zero (Zero is base value sent when not retrieving any real data from arduino)
               else if(sensors[2] <= metalSensorCheck && sensors[2] > 0 && isPlayingClip == false)
               {
                 if(secondSensTwo == false)
                 {
                   //Set the flag for the sensor event being triggered
                    secondSensTwo = true;
                    //Playing the sound clip
                    secondSpotExtraTwoFile.play();
                    //Get the initial time for the timer
                    timerStart = millis();
                    //Setting the duration to be the length of the current clip
                    duration = secondSpotExtraTwoFile.duration();
                    //Getting the duration into milliseconds
                    duration *= 1000;
                    //Set the flag for playing the clip
                    isPlayingClip = true;
                 }
               } 
            }
          //Check if the second station has not yet been activated
          //Checking this after the second checks so that those sensors will not go off the first time through
          if(!secondIsActivated)
          { 
            //Set the second station to have been activated
            secondIsActivated = true;
            //Begin playing the sound clip
            secondSpotMainFile.play();
            //Get the initial time for the timer
            timerStart = millis();
            //Setting the duration to be the time of the current sound
            duration = secondSpotMainFile.duration();
            //Extending the time to be based in milliseconds
            duration *= 1000;
            //Set the flap that a clip is currently going so that timer checks can start
            isPlayingClip = true;
          } 
        }
      }
    }
  }
  
  //If a clip is currently playing, begin checking if the clip is over
  if(isPlayingClip)
  {
    //If the start of the timer + duration is no longer larger than the current amount of milliseconds, this means that the clip has past its duration, and the clip needs to be ended
   if((timerStart + duration) <= millis())
   {
     isPlayingClip = false;
   }
  }
  println((timerStart + duration) + ", " + millis());
  //Sending data so that more will be recieved back from the arduino
  myPort.write("a");
}

Explorer Arduino

#include <SPI.h>
#include <nRF24L01.h>
#include <RF24.h>
#include <NewPing.h>
#include <stdio.h>
#include <stdlib.h>

RF24 radio(7, 8); // CE, CSN
const byte address[6] = "00069";

#define TRIGGER_PIN  5  // Arduino pin tied to trigger pin on the ultrasonic sensor.
#define ECHO_PIN     6  // Arduino pin tied to echo pin on the ultrasonic sensor.
#define MAX_DISTANCE 200 // Maximum distance we want to ping for (in centimeters). Maximum sensor distance is rated at 400-500cm.

#define BUTTON_PIN   4

NewPing sonar(TRIGGER_PIN, ECHO_PIN, MAX_DISTANCE); // NewPing setup of pins and maximum distance.

void setup() {
  Serial.begin(9600);
  pinMode(BUTTON_PIN, INPUT);
  radio.begin();
  radio.openWritingPipe(address);
  radio.setPALevel(RF24_PA_MIN);
  radio.stopListening();
}
void loop() { 
  if(digitalRead(BUTTON_PIN)) {
    delay(50);

    char text[64];
    
    // get sonar ping in microseconds
    unsigned int uS = sonar.ping(); 
  
    // add ultrasonic ping in inches
    itoa((uS / US_ROUNDTRIP_IN), text, 10);
  
    // add water sensor
    char watertxt[8] = ",";
    char waterval[8];
    itoa(analogRead(A0), waterval, 10);
    strcat(watertxt, waterval);
    strcat(text, watertxt);
  
    // add metal sensor
    char metaltxt[8] = ",";
    char metalval[8];
    itoa(analogRead(A1), metalval, 10);
    strcat(metaltxt, metalval);
    strcat(text, metaltxt);

    Serial.println(text);
      
    radio.write(&text, sizeof(text));
  } else {
    char zeroes[8] = "0,0,0";

    Serial.println(zeroes);
    delay(1);
    
    radio.write(&zeroes, sizeof(zeroes));
  }
}

Home Base Arduino

#include <SPI.h>
#include <nRF24L01.h>
#include <RF24.h>
RF24 radio(7, 8); // CE, CSN
const byte address[6] = "00069";

int distance;
int water;
int metal;

String rawInput;

void setup() {
  Serial.begin(9600);
  radio.begin();
  radio.openReadingPipe(0, address);
  radio.setPALevel(RF24_PA_MIN);
  radio.startListening();
}
void loop() {
  if (radio.available()) {
    char text[32] = "";
    radio.read(&text, sizeof(text));
    rawInput = text; // get the raw input and save it outside the loop
    Serial.println(rawInput);
    //parseInput(rawInput);
  }
  //Serial.println("radio not available");

}

We also have a link to a video of us demonstrating the prototype.
Link: https://youtu.be/xGOQRewGvl4

Overall I think we accomplished our goal although it does have technical limitations. We enable the user to explore the space and interact with different objects in order to learn about the context. The largest upgrades we could make in the future would be better location detection and the designing of a full exhibit designed to take advantage of the format.

Here are a few pictures of our "explorer".



Comments