Hand Entry of Data Logs

General Problem: Characterizing errors found in data sets.

Tip 1: ALL DATA SETS CONTAIN ERRORS!

• Sources of error:
  • Errors in hand-written logs due to distracted technican
  • Typos that occur when keying in data
  • Data placed in wrong row or column of a table
  • Data in wrong units (e.g. feet instead of meters)
  • Null (or missing) values interpreted as data
  • Sensor malfunctions
  • Data transmission/copy errors
  • ... and a million other ways things can go wrong
• Error rates of 1 error in every 10-20 rows of a table of data are typical).
• Understand where your data comes from, in as much detail as you can.
• Understand what the data values mean, so to look for anomalous behavior.
• Be on the lookout for errors:
  • dates and times that are out-of-sequence
  • values that are out of the acceptable range of a parameter
  • values that are stated to unusual precision
  • repeated sequences of entries
• Perform consistency checks, where possible.

Things to do:

1. The West Point Ozone Project Sheet 43 handout is a hand-written data log. In many data-collection projects a technician is responsible for making routine measurements and writing the results down on log sheets like this one. (Actually, this particular sheet is fake - constructed for the purpose of illustration only). Look the sheet over, noting that it contains 5 columns of data:
   1. Date, in mm/dd/yyyy format
   2. Time, in twenty-four style
   3. Wind direction, in degress east of north
   4. Wind speed, in miles per hour
   5. Atmospheric ozone concentration, in parts per billion
2. Enter this data, exactly as it appears on the sheet, into an Excel spreadsheet. Just do a first cut at data-entry. Don't go back and check your work.
3. Add two lines at the top of the spreadsheet
   1. Parameter names (e.g. data, time, etc.)
   2. Parameter units (e.g. degrees, miles/hr, etc.)
4. Print out your result, write your name on it, and exchange it with another member of the class. Check the other member's sheet for typing errors. Count up the total number of errors and report them to the instructor.
5. Get your sheet back. Of course, the technician who created the sheet may have made errors, too. Identify as many 'possible' error as you can. Should you correct them?
6. Convert wind speed from miles per hour to meters per second.