EEP - raw2cnt

raw2cnt - Generic Raw Data Import Filter

Synopsis
Description
Data File
Configuration File
Example

Synopsis


-----------------------------------------------------------------------------
EEP 3.1  Max-Planck-Institute of Cognitive Neuroscience 1996-99
raw2cnt 3.1   (OSF1 V4.0 alpha)               Wed Sep 15 13:30:12 1999
-----------------------------------------------------------------------------
  raw2cnt [options] <src file> <cfg> <dest cnt>

options:
  -t <type>    data type; one of ascii, s8, s16, s32, f32
               default: ascii
  -o <ofs>     start of data in file (in lines (-t ascii) or bytes)
               default: first line (1) or first byte (0)
  -b <block>   number of subsequent data points for one channel
               default: 1
  -l <length>  write max. <length> samples per channel
               default: 0 (convert until end of file)
  -f <freq>    sampling frequency (in Hz)
               default: 250
  -s           swap byteorder when 16/32 bit binary types are read

Description

raw2cnt is intended to ease the signal data exchange with other software. It can convert several flavours of simple data files to EEP's cnt format.

Since most software can export some kind of "plain", "ascii", "raw" or "unformatted" matrices, there is a good chance that you can import whatever data you have via raw2cnt.

Additional information (channel labels, scalings etc.) and the information how to decode the actual data is expected from a configuration file and from command line arguments.

Data File

The data file can be nearly any "plain" list of numbers. The - not so hard - requirements are:

The data values must be stored as ASCII text, 8/16/32 bit signed integer or 32 bit floating point numbers.
The data values must be arranged in a systematic way:
n values for first channel, n values for second channel, ..., n values for last channel; next n values for first channel ...
In ASCII data files, the data values must be separated using whitespaces and the numbers must be readable with scanf()'s "%f" conversion.

If the input filename is given as "-", the standard input channel is read. Hence, it is possible to have raw2cnt at the end of a conversion pipe:

   cat something | something2raw.pl | raw2cnt - some.cfg some.cnt

Configuration File

Together with some command-line options, the configutration file describes how the data values are to be interpreted. It has one section "[channels]" with one line for each channel in the input data file:

label unit raw_scale cnt_scale

label: unique, max. 10 character channel label or keyword "SKIP" (for ignoring this channel)
unit: max. 10 character physical unit string
raw_scale: a scaling factor which is multiplied with all input data values to scale these values in the allowed cnt file data range. (max. 32 bit signed integer but 16 bit signed integer (-32768..32767) is usually enough - and consumes less diskspace).
cnt_scale: the scaling factor (in unit per bit) which is stored in the cnt file.

Some remarks to the scaling: the cnt file stores

value * raw_scale

which is required to be an integral value. But when acessing data, the EEP programs scale the data with cnt_scale. Thus you will get

value * raw_scale * cnt_scale (in units).

Typically, if the input values are in real-world units, cnt_scale equals 1.0 / raw_scale and, if the input values are "raw digitized values", raw_scale is set to 1.0 and cnt_scale is set to an arbitrary value which shoud be known by the producer of such data.

Examples

ASCII Files

Below is a hypothetical example for an ASCII EEG data file, exported by whatever program.

patient    : Max
date       : 3.3.1905
channels   : 6
rate       : 250
A1     A2     A3     A4    DIGITAL  TEST
23.0   21.2   44.5   57.2  0        42
24.0   22.2   45.5   59.2  0        42
25.0   23.2   46.5   57.2  2        42
23.0   21.2   44.5   57.2  0        42
24.0   22.2   45.5   59.2  4        42
25.0   23.2   46.5   57.2  0        42

The config file to interpret the data channels could look like this:

[channels]
A1      uV   10  0.1
A2      uV   10  0.1
A3      uV   10  0.1
A4      uV   10  0.1
DIGITAL bit   1  1   
SKIP

The input file stores microvolts with one dezimal digit - multiplication with 10 makes it integral. The channel "DIGITAL" is used "as is" and "TEST" is ignored.

The command line arguments must be:
-t ascii - it's an ASCII file,
-o 6 - data start in 6th line,
-b 1 - one value per channel,
-f 250 - the sampling rate.

Hence, the conversion command could be:

raw2cnt -t ascii -o 6 -b 1 -f 250 foo.asc foo.cfg foo.cnt

Binary Files

Just to demonstrate the flexibility: convert an EEP .avr file - which is a simple binary data file...

Assume the avrview program gives us the following information:

average file data:
  condition:    singleWo  (OCHRE)
  trials:       299/61/360 (averaged/rejected/total)
  channels:     4
  rate:         250.000 Hz
  samples:      351 (-200 .. +1200 ms)
channels:
    no  label
     1  FP1       
     2  FPZ       
     3  FP2       
     4  AF7

From the .avr fileformat specification, we know that there is an 38 byte header, followed by an 16 byte per-channel header, followed by the ERP vector of the first channel, followed by a variance vector etc. The .cfg file could be

[channels]
FP1         uV  100 0.01
SKIP
FPZ         uV  100 0.01
SKIP
FP2         uV  100 0.01
SKIP
AF7         uV  100 0.01
SKIP

and the conversion command is:

raw2cnt -t f32 -o $[38+4*16] -b 351 -f 250 foo.avr foo.cfg foo.cnt