Cosmic Rays

Notebook 1

Cosmic Ray Notebook 1

This is page 1 of the Cosmic Ray notebook. Notebook page 2 is here and Notebook page 3 is still being developed on the engineering page here.

13. Progress Made. GPS Packets sent from ubox to Arduino - 2012-10-19

I was finally able to send NMEA packets from the ubox GPS module on the ERGO shield to the Arduino Ethernet and its test application. The Arduino test application reports back to Visual Studio 2012 compiler/debugger via a USB port. This is the setup:

Logic Stack - GPS in (top left), Packets out (right)

ERGO Shield - The ublox GPS chip, embedded in a RADIONOVA antenova module produces formatted GPS packets and sends them, over its USB bus, to the ublox U-Center application for display.

The ublox GPS chip also sends the packet over a local serial bus to the Ethernet Arduino board.

Arduino - A sketch, running on the ATMEGA 328 extracts the cosmic ray event time from the packet stream, formats it into a HTTP POST command and sends it through the Ethernet port to the ERGO data server for analysis.

The ATMEGA, on the Arduino card also sends the packet over its USB port to the Visual Studio 2012 application compiler/debugger running on the software development system Windows computer.

Signal Flow (Debug setup)

/*
Software serial multiple serial test

Receives from VS 2012 app (hardware serial), sends to ERGO Shield (software serial).
Receives from ERGO Shield (software serial), sends to VS 2012 app (hardware serial).

The circuit:
* RX is digital pin 5 (connect to TX of ERGO Shield)
* TX is digital pin 6 (connect to RX of ERGO Shield)

*/
#include <SoftwareSerial.h>

SoftwareSerial mySerial(5, 6); // RX, TX

void setup()
{
// Open serial communications and wait for port to open:
Serial.begin(9600);
mySerial.begin(9600);
while (!Serial) {
; // wait for serial port to connect. Needed for Leonardo only
}
Serial.println("G o o d n i g h t m o o n!");
}

void loop() // run over and over
{
if (mySerial.available())
Serial.write(mySerial.read());

if (Serial.available())
mySerial.write(Serial.read());
}

Arduino Code for the Test

12. UBX Packet with Fletcher Algorithm Checksum - 2012-10-10

Controlling the ublox GPS module from the Arduino will require creating a Fletcher Algorithm Checksum. The following code (written in Delphi) accepts a packet of data to send TUBXPacket) and converts that to a buffer of data with the embedded checksum (TBufToSend).

---------------- Packet Structure (Delphi) ----------------

Type
  TPayload = array of Byte;
  TBufToSend = array of Byte;

Type

  TUBXPacket = Record
    SynchChar1, SynchChar2: Byte;
    pkClass: Byte;
    ID: Byte;
    PayloadLengthLSB: Byte;
    PayloadLengthMSB: Byte;
    PayLoad: TPayload;
    CK_A, CK_B: Byte;
  end;

---------------- Function to Create Send Buffer (Delphi) ----------------

function TMainForm.FormUBXSendBuf(Packet: TUBXPacket): TBufToSend;
var
  CK_A, CK_B: Byte;
  i: Byte;
  Buffer: TBufToSend;
  PayloadLength: integer;
begin

  CK_A := $0;
  CK_B := $0;
  // SetLength(Buffer, Packet.PayloadLength + 8);
  PayloadLength := (Packet.PayloadLengthMSB * 10) + Packet.PayloadLengthLSB;
  SetLength(Buffer, PayloadLength + 8);
  Buffer[2] := Packet.pkClass;
  Buffer[3] := Packet.ID;
  Buffer[4] := Packet.PayloadLengthLSB;// Little Endian swap here.
  Buffer[5] := Packet.PayloadLengthMSB;

  for i := 6 to Length(Buffer) - 3 do
  begin
    Buffer[i] := Packet.PayLoad[i - 6];
  end;
  for i := 2 to Length(Buffer) - 3 do
  begin
    CK_A := CK_A + Buffer[i];
    CK_B := CK_B + CK_A;
  end;
  Buffer[0] := Packet.SynchChar1; // Add two synch chars
  Buffer[1] := Packet.SynchChar2;
  Buffer[Length(Buffer) - 2] := CK_A;
  Buffer[Length(Buffer) - 1] := CK_B;
  Result := Buffer;
end;

This code will have to be converted to 'C' for the Arduino...

11. Arduino Sketch for URL Encoding - 2012-09-22

While writing a POST operation to a local web site I realized I had to encode the data sent. The following Arduino sketch does the trick.

void setup()   
{
  Serial.begin(9600);
  Serial.print("URL Encoder");
}

char specials[] = "$&+,/:;=?@ <>#%{}|~[]`"; ///* String containing chars you want encoded */

// FUNCTIONS
static char hex_digit(char c)
{
  return "0123456789ABCDEF"[c & 0x0F];
}

char *urlencode(char *dst,char *src)
{  
  char c,*d = dst;
  while (c = *src++)
  {  
    if (strchr(specials,c))
    {  
      *d++ = '%';
      *d++ = hex_digit(c >> 4);
      c = hex_digit(c);
    }
    *d++ = c;
  }
  *d = '\0';
  return dst;
}

// MAIN
void loop()
{
  char inbuf [ ] = "08/03/12 13:38:43 00019 +152488511 -255910812 +0024043 872986386";
  char outbuf[256] ;
  urlencode(outbuf,inbuf);
  Serial.println(outbuf);
}

URL Encoding Sketch for Arduino

10. Installing Visual Studio 2012 and Arduino for Visual Studio - 2012-08-12

This is a great set of tools for developing Arduino code. Using the Microsoft Visual Studio development system (free to download) and the Visual Micro plug (also free) you can have a modern software development system for your Arduino projects that includes real time tracing and debugging. Cool.

This is a step-by-step installation for those, like me, that don't have an in depth understanding of the Microsoft user interface for Visual Studio. Think of it as VS for boys and girls and use this as a quick start guide.

All the details are on the Visual Micro web site at http://visualmicro.com/page/Arduino-Visual-Studio-Downloads.aspx

Please remember this is state of the art code for the Arduino and is subject to change. The debug module itself is a beta version and although free to download now at some point there will be a charge. After you get your system configured using this step by step procedure you should go back to the Visual Micro site and see the complete set of configuration options.

There are five major steps in the process: Install Visual Studio, Install Arduino, install the Plug In, Initial Start up, and Install Debugger module. You may already have one or more of them completed and can skip that particular one.

Install Visual Studio <top>

Snag your copy of VS 2012 directly from Microsoft. Open this Microsoft page and join the Website Spark program. There is no charge for this and it allows you to download Visual Studio 2012 and use it for three years. That seems generous enough to me. After you have your Website Spark ID use it to find your way to the 'Get Software' button:
Download VS 2012 Professional and install it. Make sure you get the right version. It will have this logo and the color is important. A green background is the 30 day evaluation version that you don't want. This is a long process so be prepared to spend the time necessary. After a couple of false starts downloading and installing the MS installer itself I finally got the correct version up and running (about an hour on a very fast cable and computer)
The first time you start VS 2012 it will ask for your default environment. Choose "C++" (you can change it later if you want to use C# or 'F').
Close VS after it starts.

Install Arduino <top>

Download Arduino - Install the Arduino development system from here.. I download directly to a folder off My Documents simply called 'Arduino'. Start it to make sure it runs correctly. You will be asked where you installed Arduino in the next step.
Test - Make sure that the Arduino development system is running. Set the board type (I use an Ethernet Arduino) and serial port (mine is connected to Comm 8).

Install Visual Micro Plug In <top>

Download Visual Micro - Install the Visual Micro for Arduino from here.

Initial Startup <top>

Restart Visual StudioO - The first time it will ask you where you installed Arduino. If your having trouble finding this entry make sure the sort by category switch is set correctly . The entry 'Arduino Application' should point to your Arduino directory.
Open a New Sketch - Perform "File | New | Arduino Project" to open a new sketch. If you don't see this menu entry recheck your setup.
Name It - Set the new sketch 'Blinky2'. Visual Studio will create a new project (Blinky2) along with the header file and the main source (Blinky2.ino). You don't have to work with the header file it is managed by the plug in from here on in.
Open - Open Blinky2.Ino and see that you can enter the standard Blinky code from the Arduino Examples and that it runs
Check - On my system the setup looks like this
Check that you have access to all the standard VS tools and functions for editing such as Ctrl+K, Ctrl+D to reformat your source code.
From this point you have access to all the standard libraries for the Arduino and can compile, load and run all the normal code for you project. If everything works so far you can install the debug module.

Install Debugger <top>

Request the Debugger - At this point in the development cycle if you wish to try the beta version email the creator with a request (beta@visualmicro.com)
Install - When you have been accepted into the beta test program download and install the Debugger upgrade. I use all the default install options to keep the confusion to a minimum.
Restart VS 2012.
Open the Arduino sketch Blinky that you constructed earlier select Blinky2 project in the Solution Explorer
Check - Your user interface should look like this:
Set the Property - Select 'VIEW |Other Windows | Property Window'. Your user interface should now look like this:
If it does not make sure the sort options are like this:
Full - Set the (micro Debug) entry to 'Full'
Set a Breakpoint - In the Blinky code (left click out in the left gutter to get a red ball.
View Breakpoints - Open the Breakpoint window by 'DEBUG | WINDOWS | Breakpoints' .
Check - Your VS 2012 setup should look about like this:
Columns - Select the 'columns' entry and turn on 'When Hit'
Run your Blinky (press the Local Windows Debugger icon ) and you should see a trace message in the Output window every time the delay is hit.
Video - For a well done video on using the more complex debugger options watch this Tutorial:

http://www.visualmicro.com/tutorials/ArduinoDebugger_Tutorial_Level1.html

<top>

9 - POSTing with a Ethernet Arduino 2012-09-03

I have been attempting to figure out how to POST to a PHP script running on an Apache server and finally found a way to do it:

<html>
<body>
Saw your POST and Your yourname is:
<?php
echo $_POST["testone"];
//echo $_REQUEST["yourname"];
?>
<br />
</body>

</html>

PHP to field the POST (file: arduino.php at server root)

/* Web client

This sketch connects to a website (http://localhost)

using an Arduino Wiznet Ethernet shield.

Circuit:

* Ethernet shield attached to pins 10, 11, 12, 13

#include <SPI.h>

#include <Ethernet.h>

// Enter a MAC address for your controller below.

// Newer Ethernet shields have a MAC address printed on a sticker on the shield

byte mac[] = { 0x90, 0xA2, 0xda, 0x00, 0x9A, 0xA1 };

IPAddress server(192,168,1,108); // Zeke

// Initialize the Ethernet client library

// with the IP address and port of the server

// that you want to connect to (port 80 is default for HTTP):

EthernetClient client;

void setup() {

// Open serial communications and wait for port to open:

Serial.begin(115200);

while (!Serial)

{

; // wait for serial port to connect. Needed for Leonardo only

}

// start the Ethernet connection:

if (Ethernet.begin(mac) == 0) {

Serial.println("Failed to configure Ethernet using DHCP");

// no point in carrying on, so do nothing forevermore:

for(;;)

;

}

// give the Ethernet shield a second to initialize:

delay(1000);

Serial.println("connecting...");

// if you get a connection, report back via serial:

if (client.connect(server, 80)) {

Serial.println("connected");

// Make a HTTP request:

// client.println("GET /search?q=arduino HTTP/1.0");

// client.println();

client.println("POST /arduino.php HTTP/1.0");

client.println("Content-Type: application/x-www-form-urlencoded");

// the Content-Length must be set to the number of characters after the double CR/LF

// client.println("Content-Length: 9\r\n");// original Works

client.println("Content-Length: 18\r\n"); //Works

client.print("testone=JamesBrown");

}

else {

// kf you didn't get a connection to the server:

Serial.println("connection failed");

}

void loop()

{

// if there are incoming bytes available

// from the server, read them and print them:

if (client.available()) {

char c = client.read();

Serial.print(c);

}

// if the server's disconnected, stop the client:

if (!client.connected()) {

Serial.println();

Serial.println("disconnecting.");

client.stop();

// do nothing forevermore:

for(;;)

;

}

Successful POST from Arduino to PHP script

8 - Hardware Development System 2012-08-08

I came across what looks like a very good Design Automation Tool called AutoTRAX Design Express:

Take a quick look at this video...

7 - Arduino Development System 2012-08-02

In item 5 (below) I attempted to start a better Arduino software development system than the standard from Arduino itself. I searched the web high and low for something like the Embarcadero Delphi system that I have become used to after a mere 30 years but is now showing its age. The closest I could find (and its a good choice) is:

Tool	Link with instructions	Cost
Microsoft Visual Studio 2010		FREE for three years
Arduino Programming IDE		Free

These two give you a modern working environment for the Arduino.

Click this thumbnail to see my setup.

Notice the compile time (0.5 seconds) and the upload is very quick as well.

I haven't tried the debugger yet.

6 - Received Beta 4 Pixel- 2012-08-01

The ERGO pixel (beta4) unit came in today. So it wants to be set up and run.

Seti.net 86

Setting up could not be easer. Just plug it in and it runs. I attached it to one of the other GPS antennas already installed for a test.

Its name on the ERGO map will be Seti.net 86. So far I cannot find it.

Found it. Open the Google Earth application at ERGO and spin it around to Southern California. Check out #86...

5 - Arduino Development System - 2012-07-31

I found my self trying to get an Arduino crypto library 'Sha' to run using the off the shelf Arduino user interface. It is very frustrating not to have the tools that I have become accustom to in the development system. To debug with the Arduino user interface 1.0.1 you have to install print statements and you cannot set breakpoints or examine values along the way. Its also very slow. I can compile, link and load 65K lines of Delphi code faster than 20 lines of Arduino code.

It was driving me nuts.

There doesn't seem to be an Embarcadero RAD Studio plug in for Arduino that I could find but I did find one for Microsoft Visual Studio 2010. Its also possible to download a free copy of Visual studio so I thought I would give it a try. If you do this be prepared for a long task. It took over three hours to download, install, merge the help files and then start running. That's a long time and I have a 22 mbs internet access and a four core Windows 7 machine. It needs patience - something I am lacking I am sure. But I have it all installed and now am running the update system which is also taking a very long time to complete. Then I have to back Zeke up and *then* I can see if I have something I can live with.

Stay tuned.

4 - Git - 2012-07-30

After installing GitHub and playing with it for a while I decided that I absolutely *hate* the Metro user interface it has. So I found my way to tortoiseGit that installs and works like the original tortoise that I used with other C# and Delphi projects. I managed to 'Pull' the crypto library (Sha) from its Git using tortoiseGit without a problem. Sha is another story.

3 - To Do List - 2012-07-30

To get started with this project I need a list of near term goals to shoot for:

Git - Much of the useful software for the Arduino seems to be collected in Git (a revision control system) so it seems I have to install it and learn to use it.
Authentication - Sending pixel data to the ERGO server must be done using cryptology to preclude spoofing. The best seems to be a hash-based message authentication code - HMAC-SHA256. I need to figure out how that works and crank up a test application. I have found an Arduino library that includes an implementation Get.
Database - I want to build a MySQL database that mimics the database at ERGO for testing. Then I will build a POST sketch for my Arduino that will post to my web site and then to the database.
Processor - I have an Ethernet Arduino up and running. I had to buy a USB to TTL converter for it from SparkFun
GPS - I have a uBlox development kit (EVK-6T) to learn the basics of GPS software and hardware

2 - Collecting Stuff- 2012-07-28

Some things I know I need:

	This little processor will form the link between the ERGO server and the GPS system. On the left is the Ethernet connection to the SETI Net router. On the right is a small USB to RS-232 (TTL output) converter connected to my development computer (Zeke).	The software that is used to program the Arduino
Ethernet Arduino
		The ublox GPS receiver (large module with the white sticker will collect local position data of the pixel and will proved a time hack output whenever a cosmic ray is detected.
ublox EVK-6T development kit	EVK-6T inside

1 - Start Cosmic Ray Notebook - 2012-07-28

Start a new engineering notebook for the Cosmic Ray project (its what your looking at).