Page Comparison

Status

colour	Red
title	TO UPDATE

At the beginning, the Apps that are being developed used mainly one Brainlife datatype: meg/fif (see the documentation on datatypes on BL). A datatype enables Apps to communicate with each other: the output of App A is the input of App B (see . Error when running a pipeline with optional files).

...

The files written by the Apps whose output is a meg/fif-override datatype are can also be present in the meg/fif datatype.

For instance, the app-maxwell-filter takes as inputs the meg/fif datatype (for the meg.fif) and also optionnaly optionaly the meg/fif-override datatype. The inputs from the meg/fif-override datatype are optional because we don’t know if an App user will run the whole pipeline on Brainlife (i.e. compute the destination.fif, then the headshape.pos, detect bad channels, and eventually run Maxwell filter), maybe he will want to run only the app-maxwell-filter and he will provide the headshape.pos and channels.tsv without computing them with the BL Apps, and these files will be directly into the meg/fif datatype.

But, maybe the user will run the pipeline from the beginning and still provide the headshape.pos and channels.tsv in the meg/fif datatype. So, for app-maxwell-filter we will have two versions of each file. So, in this case, only the files computed by the BL Apps will be taken into account.In the Python code, we first

We deal with this situation while reading the optional files in helper.py (see Create a helper.py file):

first we read the files from the meg/fif datatype:

Code Block

    # Read channelsdestination file
    channels_file# = config.pop('channels')
    channels_file_exists = False
    if channels_file is not None: 
We don't copy this file in outdir yet because this file can be given in fif-override 
    # and if os.path.exists(channels_file):
            channels_file_exists = Truewe take the fif override file by default
    if 'destination'       df_channels = pd.read_csv(channels_file, sep='\t')in config.keys():
            # Select bad channels' name
            bad_channels = df_channels[df_channels["status"] == "bad"]['name']
            bad_channels = list(bad_channels.valuesdestination = config.pop('destination')
        if destination is not #None:
Put channels.tsv bad channels in raw.info['bads']             raw.info['bads'].sort() 
            bad_channels.sort()
            # Warning message
            if raw.info['bads'] != bad_channelsif os.path.exists(destination) is False:
                user_warning_message_channelsdestination = f'BadNone
channels from the info of your MEG file are different from ' \
                                                f'those in the channels.tsv file. By default, only bad channels from channels.tsv ' \
                                                f'are considered as bad: the info of your MEG file is updated with those channels.'
                warnings.warn(user_warning_message_channels)
                dict_json_product['brainlife'].append({'type': 'warning', 'msg': user_warning_message_channels})
                raw.info['bads'] = bad_channels

...

 else:
        destination = None

then, we read the files from the meg/fif-override datatype:

Code Block

    # Read channels file
    channels_file_override_exists = Falsethe destination file
    if 'channelsdestination_override' in config.keys():  # if the App has no meg/fif-override input this key doesn't exist
        channelsdestination_file_override = config.pop('channelsdestination_override')
        # No need to test if channelsdestination_override is None, this key is only present when the app runs on BL    
        if os.path.exists(channelsdestination_file_override) is False: 
           channels_file_override =if Nonedestination is        elsenot None:
            if channels_file_exists:   # If destination from meg/fif exists         user_warning_message_channels_file = f"You provided two channels files: by default, the file written by " \but destination_override from meg/fif-override doesn't,
                # we copy it in out_dir 
                                   f"the App detecting bad channels will be used."
                warnings.warn(user_warning_message_channels_file)
                dict_json_product['brainlife'].append({'type': 'warning', 'msg': user_warning_message_channels_file})
        channels_file_override_exists = True   
        df_channels = pd.read_csv(channels_file_override, sep='\t')
        # Select bad channels' name
        bad_channels_override = df_channels[df_channels["status"] == "bad"]['name']shutil.copy2(destination, os.path.join(out_dir_name, 'destination.fif'))  # required to run a pipeline on BL
        else:
            # If baddestination_channels_override = list(bad_channels_override.values)
        # Put channels.tsv bad channels in raw.info['bads']from meg/fif-override exists, we copy it in out_dir
            # By default we  raw.info['bads'].sort() 
        bad_channels_override.sort()copy the files given in input of meg/fif-override 
       # Warning message         if raw.info['bads'] != bad_channels_override:
            user_warning_message_channels_override = f'Bad channels from the info of your MEG file are different from ' \
                                                     f'those in the channels.tsv file. By default, only bad channels from channels.tsv ' \
       shutil.copy2(destination_override, os.path.join(out_dir_name, 'destination.fif'))  # required to run a pipeline on BL 
            destination = destination_override  #                            f'are considered as bad:we overwrite the infovalue of your MEG file is updated with those channels.'
            warnings.warn(user_warning_message_channels_override)
            dict_json_product['brainlife'].append({'type': 'warning', 'msg': user_warning_message_channels_override})
            raw.info['bads'] = bad_channels_override

...

destination

We decided not to inform the App user that the optional file previously computed by an App (so a file from the meg/fif-override datatype exists, we inform the user that we will use it and we overwrite the raw.info['bads'] with this file. So, in the file mapping datatype) will overwrite the optional file linked to the meg/fif datatype because we supposed that, if the User computed a new file he doesn’t want to use the file he gave at first when the data was staged.

Example for app-maxwell-filter

In the file mapping section (displayed when registering or eding an App on Brainlife) of app-maxwell-filter, we have:

...

For meg/fif datatype input:

...

For meg/fif-overrde datatype input:

...

Info
The `config.json` keys for the `meg/fif-override` datatype don’t correspond to the name of the files they point to: it’s not an issue henceforth the file is saved with the good name when the App is run (`shutil.copy2(destination_override, os.path.join(out_dir_name, 'destination.fif'))`)

In the config.json created by BL we will have two more keys (channels_override and headshape_override) that the keys linked to the meg/fif-override datatype, but these keys won’t be present if the App is run locally because the datatype issue is specific to BL (that’s why we check if the key is present at the beginning.

Versions Compared

Old Version 7

New Version 8

Key

Example for app-maxwell-filter