mMARCH.AC: An Open-Source R/R-Markdown Package for Processing of Accelerometry Data with GGIR in Motor Activity Research Consortium for Health (mMARCH)

Wei Guo

August 18, 2022

Introduction

What is GGIR?

GGIR is an R-package to process multi-day raw accelerometer data for physical activity and sleep research. GGIR will write all output files into two sub-directories of ./meta and ./results. GGIR is increasingly being used by a number of academic institutes across the world.

What is mMARCH.AC?

mMARCH.AC is an R-package to data processing after running GGIR for accelerometer data. In detail, all necessary R/Rmd/shell files were generated for data processing after running GGIR for accelerometer data. Then in module 1, all csv files in the GGIR output directory were read, transformed and then merged. In module 2, the GGIR output files were checked and summarized in one excel sheet. In module 3, the merged data was cleaned according to the number of valid hours on each night and the number of valid days for each subject. In module 4, the cleaned activity data was imputed by the average ENMO over all the valid days for each subject. Finally, a comprehensive report of data processing was created using Rmarkdown, and the report includes few explortatory plots and multiple commonly used features extracted from minute level actigraphy data in module 5-7. This vignette provides a general introduction to mMARCH.AC.

Software Architecture

The R package mMARCH.AC has been released with an open-source GPL-3 license on CRAN, and mMARCH.AC can run on Windows and Linux. Parallel computing in Linux is recommended due to the memory requirements associated with reading in multiple of the large data files. The package contains one primary function for users which, when run, generates all necessary R/R Markdown/shell executable files for data processing after running GGIR for accelerometer data; load, read, transform and merge long activity data; examine and summarize GGIR outputs; clean the merged activity data according to the number of valid hours per night and the number of valid days per subject; activity data imputation by taking the average across the valid days for each subject; build a comprehensive report of data processing and exploratory plots; extract multiple commonly used features and study feature structure by the covariance decomposition. Figure 1 presents a flowchart for each step in this process which is described in greater detail below. The procedure, R functions, inputs, and outputs are all described in this package vignette. In addition, more documentation and example data could be found in mMARCH.AC repository on GitHub (URL: https://github.com/WeiGuoNIMH/mMARCH.AC).

Mirroring the GGIR structure of processing individual data files in multiple parts, the mMARCH.AC package is split into seven modules, grouping functionalities in logical processing order. The modules are numbered from 1 to 7. modules 1 to 4 are dedicated to data processing. modules 5 to 7 are dedicated to producing R Markdown reports of data cleaning, feature extraction, and unsupervised covariance decomposition via the joint and individual variance explained (JIVE) method, respectively. These seven modules are carried out sequentially with milestone data automatically being saved locally. To use mMARCH.AC, the first step for users is to install and load the mMARCH.AC package. Then, users run the create.shell() function which creates a single R script. The newly created R script, Studyname_module0.maincall.R, is then edited by users, allowing for the specification of arguments relevant for each of the seven modules. All optional arguments and their defaults are described in the package vignette. In addition, for users with access to a cluster for parallel processing, a shell function, named as module9_swarm.sh is created which can parallelizes the processing of individual files with minor modifications by the user. These modifications are described in the package vignette. Computationally, module 1 is the most time-consuming task, taking up at least 60% of the processing time, which the activity data in .csv format was transformed and merged. Generally, module 1 takes about 10~30 minutes to process a file with 14 days of data recorded at 30 Hz on a GeneActiv device in processor cores of 36 x 2.3 GHz (Intel Gold 6140). All output created for each module is described in the package vignette. Briefly, module 1 and module 2 output are saved using a directory structure with a depth of two, containing output data and summary for all participants. The reports for modules 5 to 7 are saved in .html format and are generated using R Markdown (.Rmd) files. These .Rmd files are included in the output, users the flexibility to adapt the source code to their research purpose.

Figure 1: Overview of main steps and output in mMARCH.AC workflow.

Software Dependencies

All mMARCH.AC code is written in R and reports generated in R Markdown. The R packages ActFrag and ActCR are used for the calculation of certain physical activity and circadian rhythmicity features. The R package r.jive is used to perform the feature interaction analysis and to study the joint and individual variation structure by JIVE.

Setting up your work environment

Install R and RStudio

Download and install R

Download and install RStudio (optional, but recommended)

Download GGIR with its dependencies, you can do this with one command from the console command line:

install.packages("mMARCH.AC", dependencies = TRUE)

Prepare folder structure

  1. folder of .bin files for GGIR or a file listing all .bin files
    • R program will check the missing in the GGIR output by comparing with all raw .bin files
  2. foder of the GGIR output with two sub-folders
    • meta (./basic, ./csv, etc)
    • results (modulesummary.csv)

Quick start

Create a template shell script of mMARCH.AC

library(mMARCH.AC) 
create.shell()

The function will create a template shell script of mMARCH.AC in the current directory, names as STUDYNAME_module0.maincall.R.

cat STUDYNAME_module0.maincall.R
 
argv = commandArgs(TRUE);  
print(argv) 
print(paste("length=",length(argv),sep=""))  
mode<-as.numeric(argv[1])  
print(c("mode =", mode))
# (Note) Please remove the above lines if you are running this within R console 
#        instead of submitting jobs to a cluster.
 
#########################################################################   
# (user-define 1) you need to redefine this according different study!!!!
######################################################################### 
# example 1 
filename2id.1<-function(x)  unlist(strsplit(x,"\\."))[1] 
 
#  example 2 (use csv file =c("filename","ggirID")) 
filename2id.2<-function(x) {
  d<-read.csv("./mMARCH.AC/inst/extdata/example/filename2id.csv",head=1,stringsAsFactors=F)
  y1<-which(d[,"filename"]==x)
  if (length(y1)==0) stop(paste("Missing ",x," in filename2id.csv file",sep=""))
  if (length(y1)>=1) y2<-d[y1[1],"newID"] 
  return(as.character(y2))
} 


#########################################################################  
#  main call
######################################################################### 
  
mMARCH.AC.shell<-function(mode,filename2id=NULL){ 

library(mMARCH.AC) 
packageVersion("mMARCH.AC")
#  ?mMARCH.AC.maincall  # run help to see all argumengts 

#########################################################################  
# (user-define 2) Fill in parameters of your ggir output
########################################################################## 
  
currentdir = 
studyname =
bindir = 
outputdir =  
setwd(currentdir) 

rmDup=FALSE   # keep all subjects in mMARCH.AC
PA.threshold=c(50,100,400)
part5FN="WW_L50M100V400_T5A5" 
epochIn = 5
epochOut =  60
use.cluster = FALSE
log.multiplier = 9250
QCdays.alpha = 7
QChours.alpha = 16 
QCnights.feature.alpha = c(0,0)
DoubleHour= "average" 
QC.sleepdur.avg=NULL
QC.nblocks.sleep.avg=NULL
useIDs.FN=NULL 
Rversion="R" 
desiredtz="US/Eastern" 
RemoveDaySleeper=FALSE  
NfileEachBundle=20 
holidayFN=NULL
trace=FALSE
#########################################################################  
#   remove duplicate sample IDs for plotting and feature extraction 
######################################################################### 
if (mode==3 & rmDup){
# step 1: read ./summary/*remove_temp.csv file (output of mode=2)
keep.last<-TRUE #keep the latest visit for each sample
sumdir<-paste(currentdir,"/summary",sep="")  
setwd(sumdir)  
inFN<-paste(studyname,"_samples_remove_temp.csv",sep="")
useIDs.FN<-paste(sumdir,"/",studyname,"_samples_remove.csv",sep="") 

#########################################################################  
# (user-define 3 as rmDup=TRUE)  create useIDs.FN file
######################################################################### 
# step 2: create the ./summary/*remove.csv file manually or by R commands
d<-read.csv(inFN,head=1,stringsAsFactors=F)
d<-d[order(d[,"Date"]),]
d<-d[order(d[,"newID"]),]
d[which(is.na(d[,"newID"])),]
S<-duplicated(d[,"newID"],fromLast=keep.last) #keep the last copy for nccr
d[S,"duplicate"]<-"remove"
write.csv(d,file=useIDs.FN,row.names=F)

} 

#########################################################################  
#   call afterggir
######################################################################### 

setwd(currentdir)  
mMARCH.AC.maincall(mode=mode,
          useIDs.FN=useIDs.FN,
          currentdir=currentdir,
          studyname=studyname,
          bindir=bindir,
          outputdir=outputdir,
          epochIn=epochIn,
          epochOut=epochOut, 
          log.multiplier=log.multiplier,
          use.cluster=use.cluster,
          QCdays.alpha=QCdays.alpha,
          QChours.alpha=QChours.alpha,
          QCnights.feature.alpha=QCnights.feature.alpha, 
          DoubleHour= DoubleHour,
          QC.sleepdur.avg=QC.sleepdur.avg,
          QC.nblocks.sleep.avg=QC.nblocks.sleep.avg,
          Rversion=Rversion,
          filename2id=filename2id,
          PA.threshold=PA.threshold,
          desiredtz=desiredtz,
          RemoveDaySleeper=RemoveDaySleeper,
          part5FN=part5FN,
          NfileEachBundle=NfileEachBundle,
          holidayFN=holidayFN,
          trace=trace) 

} 
#########################################################################
         mMARCH.AC.shell(mode)   
######################################################################### 

#   Note:   mMARCH.AC.shell(mode)
#        mode = 0 : creat sw/Rmd file
#        mode = 1 : data transform using cluster or not
#        mode = 2 : summary
#        mode = 3 : clean 
#        mode = 4 : impu
 

Edit shell script

Three places were marked as “user-define” and need to be edited by user in the STUDYNAME_module0.maincall.R file. Please rename the file by replacing your real studyname after the edition.

1. Define the function filename2id( )

This user-defined function will change the filename of the raw accelerometer file to the short ID. For example, the first example change “0002__026907_2016-03-11 13-05-59.bin” to new ID of “0002”. If you prefer to define new ID by other way, you could create a .CSV file including “filename” and “newID” at least and then defined this function as the second example. The new variable of “newID”, included in the output files, could be used as the key ID in the summary report of mMARCH.AC and be used to define the duplicate samples as well.

2. Parameters of mMARCH.AC.maincall()

User needs to define the following parameters as follows,

Variables Description
rmDup Set rmDup = TRUE if user want to remove some samples such as duplicates. Set rmDup = FALSE if user want to keep all samples.
mode Specify which of the five modules need to be run, e.g. mode = 0 makes that all R/Rmd/sh files are generated for other modules. When mode = 1, all csv files in the GGIR output directory were read, transformed and then merged. When mode = 2, the GGIR output files were checked and summarized in one excel sheet. When mode = 3, the merged data was cleaned according to the number of valid hours on each night and the number of valid days for each subject. When mode = 4, the cleaned data was imputed.
useIDs.FN Filename with or without directory for sample information in CSV format, which including “filename” and “duplicate” in the headlines at least. If duplicate=“remove”, the accelerometer files will not be used in the data analysis of module 5-7. Defaut is NULL, which makes all accelerometer files will be used in module 5-7.
currentdir Directory where the output needs to be stored. Note that this directory must exist.
studyname Specify the study name that used in the output file names
bindir Directory where the accelerometer files are stored or list
outputdir Directory where the GGIR output was stored.
epochIn Epoch size to which acceleration was averaged (seconds) in GGIR output. Defaut is 5 seconds.
epochOut Epoch size to which acceleration was averaged (seconds) in module1. Defaut is 5 seconds.
flag.epochOut Epoch size to which acceleration was averaged (seconds) in module 3. Defaut is 60 seconds.
log.multiplier The coefficient used in the log transformation of the ENMO data, i.e. log( log.multiplier * ENMO + 1), which have been used in module 5-7. Defaut is 9250.
use.cluster Specify if module1 will be done by parallel computing. Default is TRUE, and the CSV file in GGIR output will be merged for every 20 files first, and then combined for all.
QCdays.alpha Minimum required number of valid days in subject specific analysis as a quality control step in module2. Default is 7 days.
QChours.alpha Minimum required number of valid hours in day specific analysis as a quality control step in module2. Default is 16 hours.
QCnights.feature.alpha Minimum required number of valid nights in day specific mean and SD analysis as a quality control step in the JIVE analysis. Default is c(0,0), i.e. no additional data cleaning in this step.
DoubleHour Specify the method of processing the double hours for days that daylight saving time starts and ends for example. In detail, DoubleHour = c(“average”,“earlier”,“later”). The acceleration data was averaged on double hours when DoulbeHour=“average”. Only the acceleration data in the earlier occurrence was remained for double hours while the other duplicate data were ignored when DoulbeHour=“earlier”. Only the acceleration data in the later occurrence was remained for double hours while the other duplicate data were ignored when DoulbeHour=“later”. Default is “average”.
QC.sleepdur.avg As taking the deault value of QC.sleepdur.avg=c(3,12), individuals were excluded with an average sleep duration <3 hour or >12 hour.
QC.nblocks.sleep.avg As taking the deault value of QC.nblocks.sleep.avg=c(5,30), individuals were excluded with an average number of nocturnal sleep episodes <5 or >30.
Rversion R version, eg. “R/3.6.3”. Default is “R”.
filename2id User defined function for converting filename to sample IDs. Default is NULL.
PA.threshold Threshold for light, moderate and vigorous physical activity. Default is c(50,100,400).
desiredtz desired timezone: see also https://en.wikipedia.org/wiki/Zone.tab. Used in g.inspectfile(). Default is “US/Eastern”.
RemoveDaySleeper Specify if the daysleeper nights are removed from the calculation of number of valid days for each subject. Default is FALSE.
part5FN Specify which output is used in the GGIR part5 results. Defaut is “WW_L50M125V500_T5A5”, which means that part5_daysummary_WW_L50M125V500_T5A5.csv and part5_personsummary_WW_L50M125V500_T5A5.csv are used in the analysis.
NfileEachBundle Number of files in each bundle when the csv data were read and processed in a cluster. Default is 20.
trace Specify if the intermediate results is printed when the function was executed. Default is FALSE.

3. Subset of samples (optional)

The mMARCH.AC package not only simply transform/merge the activity and sleep data, but it also can do some prelimary data analysis such as principle componet analysis and feature extraction. Therefore, the basic data clean will be processed first as follows,

  • data clean by removing valid days/samples defined by minimum required number of valid hours/days in the activity data
  • remove duplicate samples

If you prefer to use all samples, just skip this module and use rmDup=FALSE as the default. Otherwise, if you want to remove some samples such as duplicates, there are two ways as follows,

  • Edit R codes of “step 2” in this module. For example, the template will keep the later copy for duplicate samples
  • Remove R codes of “step 2” in this module, and create studyname_samples_remove.csv file by filling “remove” in the “duplicate” column in the template file of studyname_samples_remove_temp.csv. The data will be kept unless duplicate=“remove”.

Run R script

mMARCH.AC.shell(mode,filename2id)   
Variables Description
mode Specify which of the five modules need to be run, e.g. mode = 0 makes that all R/Rmd/sh files are generated for other modules. When mode = 1, all csv files in the GGIR output directory were read, transformed and then merged. When mode = 2, the GGIR output files were checked and summarized in one excel sheet. When mode = 3, the merged data was cleaned according to the number of valid hours on each night and the number of valid days for each subject. When mode = 4, the cleaned data was imputed.
filename2id This user-defined function will change the filename of the raw accelerometer file to the short ID for the purpose of identifying duplicate IDs.

Run script in a cluster

#!/bin/bash
#
#$ -cwd
#$ -j y
#$ -S /bin/bash
  source ~/.bash_profile
  module load R ; 
     R --no-save --no-restore --args  < studyname_module0.maincall.R  0
     R --no-save --no-restore --args  < studyname_module0.maincall.R  1
     R --no-save --no-restore --args  < studyname_module0.maincall.R  2
     R --no-save --no-restore --args  < studyname_module0.maincall.R  3
     R --no-save --no-restore --args  < studyname_module0.maincall.R  4 
 
     R -e "rmarkdown::render('module5_studyname_Data_process_report.Rmd'   )"  
     R -e "rmarkdown::render('module6_studyname_NonWear.report.Rmd'   )" 
     R -e "rmarkdown::render('module7a_studyname_calculate_newfeatures.Rmd'   )" 
     R -e "rmarkdown::render('module7b_studyname_merge_GGIRfeatures.Rmd'   )" 
     R -e "rmarkdown::render('module7c_studyname_runJIVE.Rmd'   )" 
     R -e "rmarkdown::render('module7d_studyname_calculate_WD_WE_avg_features.Rmd'   )" 

Software Functionalities

Module 1

The functions of Module 1 read, merge, and transform the lower resolution summary of raw acceleration data for all subjects. Raw data are summarized using Euclidian Norm Minus One (ENMO) (Migueles et al., 2019), a rotationally invariant measure of volume of acceleration, in epochs specified by the user in their call to GGIR. Merged data, originally in long format, are saved in wide format, an intuitive data structure for accelerometer data (Leroux 2019). Depending on the time zone on which the devices were initialized, a day may have between 23 and 25 hours due to daylights savings time. On daylight savings crossovers, ENMO is averaged for duplicate timestamps between 1:00 AM and 2:00 AM, allowing for straightforward comparisons to standard 24-hour days. In addition to recording ENMO at each epoch, the angle of the z-axis (ANGLEZ) relative to the horizontal plane (degrees), used for estimating sleep periods, is merged and saved into an excel file. Additional data recorded by certain devices are merged for all participants and saved under the ./data directory including the light mean, light peak, temperature mean, clipping score, and the Euclidian norm metric (EN) if available.

The activity data in the GGIR output is formatted in long csv-spreadsheets as follows,
timestamp ENMO anglez
2017-11-30T00:00:00+0100 8e-04 -32.5758
2017-11-30T00:00:05+0100 0.0198 -25.5726
2017-11-30T00:00:10+0100 0.0177 3.7972
2017-11-30T00:00:15+0100 0.0118 6.7154
2017-11-30T00:00:20+0100 0.0106 10.0357
2017-11-30T00:00:25+0100 0.0341 21.0143
2017-11-30T00:00:30+0100 0.1708 19.5008
…… …… ……
2017-11-30T23:59:55+0100 0.1504 -0.596

Each row represents the corresponding ENMO and ANGLEZ values at a timestamp per 5 seconds epoch, which is specified by GGIR parameter (windowsizes) when running GGIR. After running Module 1, the ENMO and ANGLEZ data are transformed into wide matrix in which each row represents 24 hours data for a day. For example, the ENMO data is formated as follows,

Date 0:00:00 0:00:05 0:00:10 0:00:15 0:00:20 0:00:25 0:00:30 …… 23:59:55
11/30/2017 8.00E-04 0.0198 0.0177 0.0118 0.0106 0.0341 0.1708 …… 0.1504

Finally, the data was merged for all days and all subjects.

Module 2

The functions of Module 2 create descriptive variables of all accelerometer files output by GGIR. In Module 2, an excel file is output under the ./summary directory, which includes ten pages as follows, (1) List of files in the GGIR output (2) Summary of numbers of output files (3) List of duplicate IDs (4) ID errors (5) Number of valid days (6) Table of number of valid/missing days (7) Missing pattern (8) Frequency of the missing pattern (9) Description of all accelerometer files (10) device information. Multiple plots are generated in a pdf file including the number of valid hours, days, missing pattern, etc. First, the completeness of GGIR run is inspected by detecting the missing files in each GGIR step when user specified the path of GGIR output. Additionally, the descriptive variables are created to form a comprehensive data quality report based on summary results generated by GGIR for the purpose of examining data quality in the Module 2 of mMARCH.AC.

Module 3 and Module 4

The functions of Module 3 introduce ‘flag’ variables for data cleaning of the merged ENMO and ANGLEZ data. By default, days with more than 16 hours are marked as valid days and subjects with more than 7 valid days are marked as valid samples. Users can set these two parameters in the call to the main function (QCdays.alpha = 7, QChours.alpha = 16). Further, the data can be aggregated (averaged) in lower resolution epochs (e.g. minute-level or hour-level) as desired by users. In Module 4, ENMO data during estimated non-wear periods are imputed by taking the subject-level mean over all the valid days for each subject at that time, limiting potential bias in imputation due to diurnal patterns of activity. All output is saved under the ./data directory. In the output file, the following description variables are included: (1) number of valid hours; (2) missing pattern for each subject; (3) non-wear time in minutes; (4) an indicator variable to indicate if the visit should be removed for having multiple visits for some subjects which might lead to invalidity of independent and identically distribution in statistics and (5) the number of missing values after imputation. When the number of missing values after imputation is not zero, it means the activity data is missing on same timestamps among all days and therefore could not be imputed, therefore, such samples would be removed when systematic missingness was observed.

Module 5

The functions of Module 5 generate a comprehensive report in .html format. This report includes data quality checks and an exploratory data analysis using valid days of data. First, the numbers and missingness of GGIR output files are summarized for all GGIR parts. Second, duplicate samples are checked and marked, where the duplication might be caused by having multiple visits for some samples but only one visit will be kept in the data analysis such as functional principal component analysis (FPCA). Third, as shown in Figure 1, data quality is presented visually using the number of valid days, non-wear time, and missing data pattern. As an exploratory analysis, the data correlation, and the output of FPCA analysis are plotted in the report.

Module 6 (Optional)

Estimated non-wear periods are loaded from the \(M\$metalong\$nonwearscore\) variable of the R data that was stored in the folder of /meta/basic of the GGIR output, which generate the matrix to clarify when data was imputed for each long epoch time window and the reason for imputation. This function will generate a non-wear matrix at minute level, coded as 0/1 for wear/non-wear time. This function will generate a non-wear matrix at minute level, and it could be skipped if user chose to use the imputation data in the JIVE application as default.

Module 7

In Module 7, 88 features were extracted from minute level activity data of three domains of sleep, physical activity, and circadian rhythmicity, which were based on outputs from GGIR v2.4.0 and calculated by R ActFrag and ActCR packages. The standard deviation across days on each subject was also created for each feature. The weekday and weekend specific features were extracted as well since most features in the sleep and physical activity showed significant difference between weekdays (Monday to Friday) and weekends (Saturday and Sunday). Specially, the sleep features on weekends are calculated based on sleep periods from Friday night to Saurday morning, Saturday night to Sunday morning, and last night to a holiday morning. Furthermore, holdiays could be counted as weekends if user provide the holiday information through the argument of holidayFN in the afterggir() function. In brief, sleep domain referred to sleep duration, midpoint, efficiency, etc. Physical activity referred to daily motor activity such as sedentary behavior, light, and moderate-to-vigorous physical activity (MVPA). Circadian rhythms were natural rhythms that regulates the sleep-wake cycle within every 24 hours. For example, the cosinor curve and FPCA analysis were used in modeling of biological rhythms. A comprehensive list of all features could be found in the supplementary table and more detailed definition could be found in the mMARCH.AC paper, GGIR manual and Di et al.’s publication in 2019.

All features listed are calculated using minute-level activity data by default. Several duplicate features were also extracted from GGIR results as an additional data source. As an integral module of mMARCH.AC package, we proposed to use a multi-modal integrative technique, Joint and Individual Variance Explained (JIVE), to efficiently and simultaneously deal with multiple features representing physical activity (PA), sleep (SL), and circadian rhythmicity (CR) (Di et al., 2019; Lock et al., 2013; O’Connell and Lock, 2016). The integrative analysis of multiple derived variables in each of these three domains assessed by accelerometry is applied to explain the joint and individual variation in these three domains. Conceptually, JIVE assumes four groups of latent variables that generate: i) individual variation captured by PA, SL, CR; and ii) joint variation by PA, SL, CR. These latent variables (JIVE scores) can be estimated and then used as novel measures capturing both individual and joint variability of measures of sleep, physical activity and circadian rhythmicity. JIVE has been applied to multivariate actigraphy derived features to address the interdependence between domains of physical activity, sleep, and circadian rhythm (Di et al. 2019). JIVE decomposition utilized the subject level means of all 88 features including those duplicated features from GGIR outputs. Heatmaps of the estimated joint structure of the full JIVE decomposition are created, with examples shown in Figure 1. JIVE scores are estimated for both joint and individual variation structures. Building the activity features could enrich the markers for understanding genetic contribution of human diseases.

Running mMARCH.AC and Inspecting the results

Input and output of Module 0

Output Description
module1a_data.transform.R (use.cluster=TRUE, optional) R code for data transformation and merge for every 20 files in each partition. When the number of .bin files is large ( > 1000), the data merge could take long time, user could split the job and submit the job to a cluster for parallel computing.
module1b_data.transform.sw (use.cluster=TRUE, optional) Submit the job to a cluster for parallel computing
module1c_data.transform.merge.sw (use.cluster=TRUE, optional) Merge all partitions for the ENMO and ANGLEZ data
module5_studyname_Data_process_report.Rmd R markdown file for generate a comprehensive report of data processing and explortatory plots.
module6_studyname_NonWear.report.Rmd R markdown file for generate a report of nonwear score.
module7a_studyname_calculate_newfeatures.Rmd Extract some features from the actigraphy data using R
module7b_studyname_merge_GGIRfeatures.Rmd Extract other features from the GGIR output and merge all features together
module7c_studyname_runJIVE.Rmd Perform JIVE Decomposition for All Features using r.jive
module7d_studyname_calculate_WD_WE_avg_features.Rmd Extract some weekday/weekend specific features from the actigraphy data using R
module9_swarm.sh shell script to submit all jobs to the cluster

Input and output of Module 1

Output Description
studyname_filesummary_csvlist.csv File list in the ./csv folder of GGIR
studyname_filesummary_Rdatalist.csv File list in the ./basic folder of GGIR
All_studyname_ANGLEZ.data.csv Raw data of ANGLEZ after merge
All_studyname_ENMO.data.csv Raw data of ENMO after merge
nonwearscore_studyname_f0_f1_Xs.csv Data matrix of nonwearscore
nonwearscore_studyname_f0_f1_Xs.pdf Plots for nonwearscore
plot.nonwearVSnvalidhours.csv Nonwear data for plot
plot.nonwearVSnvalidhours.pdf Nonwear plots
lightmean_studyname_f0_f1_Xs.csv Data matrix of lightmean
lightpeak_studyname_f0_f1_Xs.csv Data matrix of lightpeak
temperaturemean_studyname_f0_f1_Xs.csv Data matrix of temperaturemean
clippingscore_studyname_f0_f1_Xs.csv Raw data of clippingscore
EN_studyname_f0_f1_Xs.csv Data matrix of EN

f0 and f1 are the file index to start and finish with
Xs is the epoch size to which acceleration was averaged (seconds) in GGIR output

Input and output of Module 2

Output Description
studyname_ggir_output_summary.xlsx Description of all accelerometer files in the GGIR output. This excel file includs 9 pages as follows, (1) List of files in the GGIR output (2) Summary of files (3) List of duplicate IDs (4) ID errors (5) Number of valid days (6) Table of number of valid/missing days (7) Missing patten (8) Frequency of the missing pattern (9) Description of all accelerometer files.
part2daysummary.info.csv Intermediate results for description of each accelerometer file.
studyname_ggir_output_summary_plot.pdf Some plots such as the number of valid days, which were included in the module5_studyname_Data_process_report.htm file as well.
studyname_samples_remove_temp.csv Create studyname_samples_remove.csv file by filling “remove” in the “duplicate” column in this template. If duplicate=“remove”, the accelerometer files will not be used in the data analysis of module5.

Input and output of Module 3

Output Description
flag_All_studyname_ANGLEZ.data.Xs.csv Adding flags for data cleaning of the raw ANGLEZ data
flag_All_studyname_ENMO.data.Xs.csv Adding flags for data cleaning of the raw ENMO data
IDMatrix.flag_All_studyname_ENMO.data.60s.csv ID matrix

*Xs is the epoch size to which acceleration was averaged (seconds) in GGIR output

Input and output of Module 4

Output Description
impu.flag_All_studyname_ENMO.data.60s.csv Imputation data for the merged ENMO data, and the missing values were imputated by the average ENMO over all the valid days for each subject.

Description of flag variables in the output data

Variable Description
filename accelerometer file name
Date date recored from the GGIR part2.summary file
id IDs recored from the GGIR part2.summary file
calender_date date in the format of yyyy-mm-dd
N.valid.hours number of hours with valid data recored from the part2_daysummary.csv file in the GGIR output
N.hours number of hours of measurement recored from the part2_daysummary.csv file in the GGIR output
weekday day of the week-Day of the week
measurementday day of measurement-Day number relative to start of the measurement
newID new IDs defined as the user-defined function of filename2id(), e.g. substrings of the filename
missing “M” indicates missing for an invalid day, and “C” indicates completeness for a valid day
Ndays number of days of measurement
ith_day rank of the measurementday, for example, the value is 1,2,3,4,-3,-2,-1 for measurementday = 1,…,7
Nmiss number of missing (invalid) days
Nnonmiss number of non-missing (valid) days
misspattern indicators of missing/nonmissing for all measurement days at the subject level
RowNonWear number of columnns in the non-wearing matrix
NonWearMin number of minutes of non-wearing
remove16h7day indicator of a key qulity control output. If remove16h7day=1, the day need to be removed. If remove16h7day=0, the day need to be kept.
duplicate If duplicate=“remove”, the accelerometer files will not be used in the data analysis of module5.
ImpuMiss.b number of missing values on the ENMO data before imputation
ImpuMiss.a number of missing values on the ENMO data after imputation
KEEP The value is “keep”/“remove”, e.g. KEEP=“remove” if remove16h7day=1 or duplicate=“remove” or ImpuMiss.a>0

Input and output of Module 5

Output Description
module5_studyname_Data_process_report.html A comprehensive report of data processing and explortatory plots.

Input and output of Module 6

Folder Output Description
./ module6_studyname_NonWear.report.html A report of nonwear score.
./data JIVEraw_nonwearscore_studyname_1_5_Xs.csv Imputation data matrix of nonwearscore (1/0)
./data JIVEimpu_nonwearscore_studyname_1_5_Xs.csv Data matrix of nonwearscore (1/0/NA)

f0 and f1 are the file index to start and finish with
Xs is the epoch size to which acceleration was averaged (seconds) in GGIR output

Input and output of Module 7a

Output Description
module7_studyname_all_features_dictionary.xlsx Description of features
module7a_studyname_calculate_newfeatures.html Extract some features from the actigraphy data using R
module7a_studyname_new_features_page1_features.csv List of some features
module7a_studyname_new_features_page2_face_day_PCs.csv Function PCA at the day level using fpca.face( )
module7a_studyname_new_features_page3_face_subject_PCs.csv Function PCA at the subject level using fpca.face( )
module7a_studyname_new_features_page4_denseFLMM_day_PCs.csv Function PCA at the day level using denseFLMM( )
module7a_studyname_new_features_page5_denseFLMM_subject_PCs.csv Function PCA at the subject level using denseFLMM( )

Input and output of Module 7b

Output Description
module7b_studyname_merge_GGIRfeatures.html Extract other features from the GGIR output and merge all features together
module7b_studyname_all_features_1_day.csv Raw data of all features
module7b_studyname_all_features_2_dayclean.csv Keep sample with valid ENMO inputs
module7b_studyname_all_features_2_dayclean.csv.log Log file of each variable of module5b_studyname_all_features_2_dayclean.csv
plot_module7b_studyname_all_features_2_dayclean.csv.pdf Plot of each variable of module5b_studyname_all_features_2_dayclean.csv
module7b_studyname_all_features_3_subject.csv Average variable at the subject level
module7b_studyname_all_features_3_subject.csv.log Log file of each variable of module5b_studyname_all_features_3_subject.csv
plot_module7b_studyname_all_features_3_subject.csv.pdf Plot of each variable of module5b_studyname_all_features_3_subject.csv
module7b_studyname_all_features_4_subjectSD.csv subject level SD of each feature

Input and output of Module 7c

Output Description
module7c_studyname_runJIVE.html Perform JIVE Decomposition for All Features using r.jive
module7c_studyname_jive_Decomposition.csv Joint and individual structure estimates
module7c_studyname_jive_predScore.csv PCA scores of JIVE ( missing when jive.predict failes)

Input and output of Module 7d

Output Description
module7d_studyname_calculate_WD_WE_avg_features.html Extract some weekday/weekend specific features from the actigraphy data using R
module7d_studyname_new_features_page1.csv Perform JIVE Decomposition for All Features using r.jive
module7d_weekday_studyname_all_features_3_subject.csv subject level mean of each feature on weekday
module7d_weekday_studyname_new_features_page4_denseFLMM_day_PCs.csv Function PCA at the day level using denseFLMM( ) on weekday
module7d_weekday_studyname_new_features_page5_denseFLMM_subject_PCs.csv Function PCA at the subject level using denseFLMM( ) on weekday
module7d_weekend_studyname_all_features_3_subject.csv subject level mean of each feature on weekend
module7d_weekend_studyname_new_features_page4_denseFLMM_day_PCs.csv Function PCA at the day level using denseFLMM( ) on weekend
module7d_weekend_studyname_new_features_page5_denseFLMM_subject_PCs.csv Function PCA at the subject level using denseFLMM( ) on weekend

Description of features of domains of physical activity, sleep and circadian rhythmicity

Sleep Domain

Variable Description level Source
sleeponset Detected onset of sleep expressed as hours since the midnight of the previous night. day GGIR-part4
wakeup Detected waking time (after sleep period) expressed as hours since the midnight of the previous night. day GGIR-part4
number_sib_wakinghours Number of sustained inactivity bouts during the day, with day referring to the time outside the Sleep Period Time window. day GGIR-part4
SleepDurationInSpt Total sleep duration, which equals the accumulated nocturnal sustained inactivity bouts within the Sleep Period Time day GGIR-part4
sleep_Midpoint Sleep Midpoint = (sleeponset + wakeup)/2. day GGIR-part4
sleep_efficiency Sleep Efficiency = SleepDurationInSpt / (wakeup - sleeponset), which was the fraction of time spent asleep in the Sleep Period Time window. day GGIR-part4
N_atleast5minwakenight Number of times awake during the night for at least 5 minutes day GGIR-part5
ACC_spt_sleep_mg Average acceleration during sleep (mg) day GGIR-part5
Nblocks_spt_sleep Number of blocks of night sleep within the Sleep period time window. day GGIR-part5

Physical Activity Domain

Variable Description level Source
TAC Total volume of physical activity in the daytime day mMARCH.AC
TLAC Total volume of log-transformed physical activity in the daytime day mMARCH.AC
TAC_24h Total volume of physical activity within 24 hours day mMARCH.AC
TLAC_24h Total volume of log-transformed physical activity within 24 hours day mMARCH.AC
sed_dur Total daytime duration in minutes of sedentary activity day mMARCH.AC
light_dur Total daytime duration in minutes of light activity day mMARCH.AC
mod_dur Total daytime duration in minutes of moderate activity day mMARCH.AC
vig_dur Total daytime duration in minutes of vigorous activity day mMARCH.AC
MVPA_dur Total daytime duration in minutes of moderate to vigorous activity day mMARCH.AC
sed_dur_M10 Total duration in minutes of sedentary activity within M10 window day mMARCH.AC
light_dur_M10 Total duration in minutes of light activity within M10 window day mMARCH.AC
mod_dur_M10 Total duration in minutes of moderate activity within M10 window day mMARCH.AC
vig_dur_M10 Total duration in minutes of vigorous activity within M10 window day mMARCH.AC
MVPA_dur_M10 Total duration in minutes of moderate to vigorous activity within M10 window day mMARCH.AC
sed_dur_24h Total duration in minutes of sedentary activity within 24 hours day mMARCH.AC
light_dur_24h Total duration in minutes of light activity within 24 hours day mMARCH.AC
mod_dur_24h Total duration in minutes of moderate activity within 24 hours day mMARCH.AC
vig_dur_24h Total duration in minutes of vigorous activity within 24 hours day mMARCH.AC
MVPA_dur_24h Total duration in minutes of moderate to vigorous activity within 24 hours day mMARCH.AC
mean_r Mean sedentary bout duration. day mMARCH.AC
mean_a Mean active bout duration. day mMARCH.AC
SATP Sedentary to active transition probabilities. day mMARCH.AC
ASTP Active to sedentary transition probabilities. day mMARCH.AC
Gini_r Gini index for active bout, absolute variability normalized to the average bout duration (resting). day mMARCH.AC
Gini_a Gini index for sedentary bout, absolute variability normalized to the average bout duration (active). day mMARCH.AC
alpha_r Power law parameter for sedentary bout. day mMARCH.AC
alpha_a Power law parameter for active bout. day mMARCH.AC
h_r Hazard function for sedentary bout. day mMARCH.AC
h_a Hazard function for active bout. day mMARCH.AC
dur_day_total_IN_min Total duration of day in minutes spent in total inactivity during the day day GGIR-part5
dur_day_total_LIG_min Total duration of day in minutes of light activity during the day day GGIR-part5
dur_day_total_MOD_min Total duration of day in minutes of moderate activity during the day day GGIR-part5
dur_day_total_VIG_min Total duration of day in minutes of vigorous activity during the day day GGIR-part5
dur_day_MVPA_bts_10_min Total duration in minutes of Moderate and Vigorous Physical Activity (MVPA) for bouts 10 minutes or more day GGIR-part5
dur_day_MVPA_bts_5_10_min Total duration in minutes of Moderate and Vigorous Physical Activity (MVPA) for bouts 5 to 10 minutes day GGIR-part5
dur_day_MVPA_bts_1_5_min Total duration in minutes of Moderate and Vigorous Physical Activity (MVPA) for bouts 1 to 5 minutes day GGIR-part5
Nbouts_day_IN_bts_30 Number of bouts of inactivity for bouts 30 minutes or more day GGIR-part5
Nbouts_day_IN_bts_20_30 Number of bouts of inactivity for bouts 20 to 30 minutes day GGIR-part5
Nbouts_day_IN_bts_10_20 Number of bouts of inactivity for bouts 10 to 20 minutes day GGIR-part5
Nbouts_day_LIG_bts_10 Number of bouts of light for bouts 10 minutes or more day GGIR-part5
Nbouts_day_LIG_bts_5_10 Number of bouts of light for bouts 5 to 10 minutes day GGIR-part5
Nbouts_day_LIG_bts_1_5 Number of bouts of light for bouts 1 to 5 minutes day GGIR-part5
Nbouts_day_MVPA_bts_10 Number of bouts of Moderate and Vigorous Physical Activity (MVPA) for bouts 10 minutes or more day GGIR-part5
Nbouts_day_MVPA_bts_5_10 Number of bouts of Moderate and Vigorous Physical Activity (MVPA) for bouts 5 to 10 minutes day GGIR-part5
Nbouts_day_MVPA_bts_1_5 Number of bouts of Moderate and Vigorous Physical Activity (MVPA) for bouts 1 to 5 minutes day GGIR-part5
Nblocks_day_total_IN Number of blocks of total inactivity during day day GGIR-part5
Nblocks_day_total_LIG Number of blocks of total light activity during day day GGIR-part5
Nblocks_day_total_MOD Number of blocks of total moderate activity during day day GGIR-part5
Nblocks_day_total_VIG Number of blocks of total vigorous activity during day day GGIR-part5

Circadian Rhythmicity Domain

Variable Description level Source
L5TIME_num Timing of least active 5 hours day GGIR-part5
M10TIME_num Timing of most active 10 hours day GGIR-part5
L5VALUE Average acceleration value (mg) of least active 5 hours day GGIR-part5
M10VALUE Average acceleration value (mg) of most active 10 hours day GGIR-part5
RA_ggir Relative amplitude reflects the difference between M10 activity and L5 activity and RA_ggir = (M10VALUE-L5VALUE)/(M10VALUE+L5VALUE). day GGIR-part5
L5 Average acceleration value (mg) of least active 5 hours day mMARCH.AC
M10 Average acceleration value (mg) of most active 10 hours day mMARCH.AC
L5TIME Timing of least active 5 hours day mMARCH.AC
M10TIME Timing of most active 10 hours day mMARCH.AC
RA Relative amplitude: (M10-L5)/(M10+L5) where M10= most active 10 hrs, L5 = least active 5 hour. day mMARCH.AC
IV Intra-daily variability measures fragmentation in the rest/activity rhythms. day mMARCH.AC
IS Inter-daily stability measures fragmentation in the rest/activity rhythms between different days. subject mMARCH.AC
IV_intradailyvariability Intra-daily variability measures fragmentation in the rest/activity rhythms. subject GGIR-part2
IS_interdailystability Inter-daily stability measures fragmentation in the rest/activity rhythms between different days. subject GGIR-part2
mesor MESOR which is short for midline statistics of rhythm, which is a rhythm adjusted mean. This represents mean activity level. subject mMARCH.AC
amp Amplitude, a measure of half the extend of predictable variation within a cycle. This represents the highest activity one can achieve. subject mMARCH.AC
acro Acrophase, a measure of the time of the overall high values recurring in each cycle. Here it has a unit of radian. This represents time to reach the peak. subject mMARCH.AC
mesor_L Mesor in the cosinor model by using log-transformed data. subject mMARCH.AC
amp_L Amplitude log-transformed in the cosinor model by using log-transformed data. subject mMARCH.AC
acro_L Acrophase log-transformed in the cosinor model by using log-transformed data. subject mMARCH.AC
PC1 1st principal component score from functional principal component analysis (FPCA). subject mMARCH.AC
PC2 2nd principal component score from functional principal component analysis (FPCA). subject mMARCH.AC
PC3 3rd principal component score from functional principal component analysis (FPCA). subject mMARCH.AC
PC4 4th principal component score from functional principal component analysis (FPCA). subject mMARCH.AC
PC5 5th principal component score from functional principal component analysis (FPCA). subject mMARCH.AC
PC6 6th principal component score from functional principal component analysis (FPCA). subject mMARCH.AC
PC7 7th principal component score from functional principal component analysis (FPCA). subject mMARCH.AC
PC8 8th principal component score from functional principal component analysis (FPCA). subject mMARCH.AC
PC9 9th principal component score from functional principal component analysis (FPCA). subject mMARCH.AC
PC10 10th principal component score from functional principal component analysis (FPCA). subject mMARCH.AC

Note
(1) The column of level indicates the feature was calculated as subject level or day level.
(2) The column of source indicates the feature source. R indiciates the feature was calculated based on activity data. Part2, part4 and part5 in ‘Source’ column indicate the feature was extracted from summary results from GGIR outputs.

Citing mMARCH.AC

Reference

Websites

Please contact to ask questions.