{"id":99,"date":"2013-08-01T14:31:40","date_gmt":"2013-08-01T13:31:40","guid":{"rendered":"http:\/\/blogs.kent.ac.uk\/psychotech\/?p=99"},"modified":"2013-08-01T14:43:54","modified_gmt":"2013-08-01T13:43:54","slug":"data-programming-support-for-working-with-large-data-files","status":"publish","type":"post","link":"https:\/\/blogs.kent.ac.uk\/hsstech\/2013\/08\/01\/data-programming-support-for-working-with-large-data-files\/","title":{"rendered":"DATA: Programming support for working with large data files"},"content":{"rendered":"

Summary:<\/span><\/p>\n

Increasingly, research systems used in Psychology are generating large volumes of data (e.g. eye trackers, eeg,\u2026) The Psychology Technical group have recently provided programming support to help with processing eye-tracking and Brain Vision data and are always happy to help out, if possible by writing bespoke programs.<\/p>\n

Contact us on x7995 or at psychsupport@kent.ac.uk<\/a> to arrange a time to discuss your data and whether we might be able to help.<\/p>\n

See below for more details of a couple of recent projects<\/p>\n

Detailed:<\/span><\/p>\n

Example 1 \u2013 Eye tracking pupil size data<\/span><\/p>\n

With 28 data files, each with ~70,000 lines of pupil-size data looking like this:<\/p>\n


\n1\u00a0\u00a013\u00a0\u00a0 Neutral\u00a0\u00a0\u00a0180\u00a0\u00a0 162.94\u00a0\u00a0\u00a0\u00a0fixation\u00a0\u00a0 2.3834\u00a0\u00a0\u00a02.396 0\u00a0\u00a00
\n1\u00a0\u00a013\u00a0\u00a0 Neutral\u00a0\u00a0\u00a0180\u00a0\u00a0 179.69\u00a0\u00a0\u00a0 fixation\u00a0\u00a0 2.321\u00a0\u00a0\u00a0 2.383 0\u00a0\u00a04
\n1\u00a0\u00a013\u00a0\u00a0 Neutral\u00a0\u00a0 180\u00a0\u00a0 196.32\u00a0\u00a0\u00a0 fixation\u00a0\u00a0 2.4191\u00a0\u00a0 2.374 0\u00a0 0
\n1\u00a0 13\u00a0\u00a0 Neutral\u00a0\u00a0 180\u00a0\u00a0 212.94\u00a0\u00a0\u00a0 stimulus\u00a0\u00a0 2.3266\u00a0\u00a0\u00a02.401 0\u00a0\u00a04
\n1\u00a0 13\u00a0\u00a0\u00a0Neutral\u00a0\u00a0 180\u00a0\u00a0 229.57\u00a0\u00a0\u00a0\u00a0stimulus\u00a0\u00a0 2.3441\u00a0\u00a0\u00a02.298 0\u00a0 0
\n<\/code><\/p>\n

A program was written to process each file, one at a time to. For each file, baseline average values were calculate for the 500ms period pre-stimulus and then the minimum value, post stimulus was identified and further 500ms period averages was calculated with the summary results saved in separate results files. This program was written in the Python programming language. The program took less than a minute to process all the data files and was written in a manner so that it’s straightforward to modify the analysis performed so it’s ideal for situations where you’re not sure exactly how you want to analyse your data and where you need to explore it first.<\/p>\n

 <\/p>\n

Example 2 \u2013 Brain Vision \u2013 mass, automated editing of trigger files<\/span><\/p>\n

Upon opening BrainVision (BV), the trigger file (.vmrk<\/code>) is read and integrated with the raw data file (.eeg).\u00a0 However, once this has been done, the trigger file is no longer used and the only way to access it again is to delete the history files, including ICAs and artifact rejections.\u00a0 If triggers need updating without loosing your history, a compatible .csv <\/code>file, a format entirely different from the .vmrk file, must be imported.<\/p>\n

A program was written that will created new versions of the VMRK (xml) files on a list of trigger names and times taken from a separate, updated csv<\/code> file [ created as follows: \u00a0the triggers from an export file in .csv<\/code> format can be modified in excel (either by hand or algorithm) ].<\/p>\n

One example is needing to update all triggers, for all participants, with their behavioural data so as to indicate whether each trial was answered correctly or incorrectly, or even to indicate a particular response time window.\u00a0 Using code, this modification can be performed in seconds instead of days.<\/p>\n

Example csv<\/code> file extract:<\/p>\n

Blink,1,
\nS\u00a0 9,2723,
\nS101,3481,
\nS\u00a0 7,4748,
\nS 81,5515,<\/code><\/p>\n

Example VMRK file extract:
\n
\n<Marker>
\n<Type>Stimulus<\/Type>
\n<Description>S\u00a0 8<\/Description>
\n<Position>33698<\/Position>
\n<Points>1<\/Points>
\n<Channel>All<\/Channel>
\n<Date>0001-01-01T00:00:00<\/Date>
\n<\/Marker><\/code><\/p>\n

 <\/p>\n

If you have any data analysis questions, contact us on x7995 or at psychsupport@kent.ac.uk<\/a> to arrange a time to discuss your data and whether we might be able to help.<\/p>\n","protected":false},"excerpt":{"rendered":"

Summary: Increasingly, research systems used in Psychology are generating large volumes of data (e.g. eye trackers, eeg,\u2026) The Psychology Technical group have recently provided programming … Read more<\/a><\/p>\n","protected":false},"author":2057,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.kent.ac.uk\/hsstech\/wp-json\/wp\/v2\/posts\/99"}],"collection":[{"href":"https:\/\/blogs.kent.ac.uk\/hsstech\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.kent.ac.uk\/hsstech\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.kent.ac.uk\/hsstech\/wp-json\/wp\/v2\/users\/2057"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.kent.ac.uk\/hsstech\/wp-json\/wp\/v2\/comments?post=99"}],"version-history":[{"count":8,"href":"https:\/\/blogs.kent.ac.uk\/hsstech\/wp-json\/wp\/v2\/posts\/99\/revisions"}],"predecessor-version":[{"id":107,"href":"https:\/\/blogs.kent.ac.uk\/hsstech\/wp-json\/wp\/v2\/posts\/99\/revisions\/107"}],"wp:attachment":[{"href":"https:\/\/blogs.kent.ac.uk\/hsstech\/wp-json\/wp\/v2\/media?parent=99"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.kent.ac.uk\/hsstech\/wp-json\/wp\/v2\/categories?post=99"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.kent.ac.uk\/hsstech\/wp-json\/wp\/v2\/tags?post=99"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}