audrey06100 commited on
Commit
d91102e
·
1 Parent(s): 3566452
Files changed (1) hide show
  1. app.py +109 -167
app.py CHANGED
@@ -6,41 +6,44 @@ import random
6
 
7
  readme = """
8
 
9
- ## Introduction
10
- This tool is designed to denoise your EEG data using our pre-trained models: **ART**, **IC-U-Net**, **IC-U-Net++**, and **IC-U-Net-Attn**. The models was trained using the EEG signals of 30 channels, including: ``Fp1, Fp2, F7, F3, Fz, F4, F8, FT7, FC3, FCz, FC4, FT8, T7, C3, Cz, C4, T8, TP7, CP3, CPz, CP4, TP8, P7, P3, Pz, P4, P8, O1, Oz, O2``.So it is essential to map your channels to our template channels before using the models. This guide will walk you through the steps of mapping your channels, ensuring proper alignment with our template, and preparing your data for optimal processing.
 
 
 
 
 
 
 
11
 
12
  ## 1. Channel Mapping
13
- The following steps will guide you through the process of mapping your EEG channels to our template channels, to ensure compatibility before denoising.
14
 
15
  ### Step1: Initial Matching and Rescaling
16
  After clicking on ``Mapping`` button, we will first match your channels to our template channels by their names. Using the matched channels as reference points, we will apply Thin Plate Spline (TPS) transformation to rescale your montage to align with our template's scale. The template montage and your rescaled montage will be displayed side by side for comparison. Channels that do not have a match in our template will be **highlighted in red**.
17
- - If your data includes all the 30 template channels, you can proceed directly to **run the models**.
18
  - If your data doesn't include all the 30 template channels and you have some channels that do not match the template, you will be directed to **Step2**.
19
  - If all your channels are included in our template but you have fewer than 30 channels, you will be directed to **Step3**.
20
 
21
  ### Step2: Forwarding Unmatched Channels
22
  In this step, you will handle the channels that didn't have a direct match with our template, by manually assigning them to the template channels that are still empty, ensuring the most efficient use of your data.
23
  Your unmatched channels, previously highlighted in red, will be shown on your montage with a radio button displayed above each. You can choose to forward the data from these unmatched channels to the empty template channels. The interface will display each empty template channel in sequence, allowing you to select which of your unmatched channels to forward.
24
- - If all empty template channels are filled by your selections, you can proceed to **run the models**.
25
  - If there are still empty template channels remaining, you will be directed to **Step3**.
26
 
27
  ### Step3: Filling Remaining Template Channels
28
  To run the models successfully, we need to ensure that all 30 template channels are filled. In this step, you are required to select one of the methods provided below to fill the remaining empty template channels:
29
  - **Mean** method: Each empty template channel is filled with the average value of data from the nearest input channels. By default, the 4 closest input channels (determined after aligning your montage to the template's scale using TPS) are selected for this averaging process. On the interface, you will see checkboxes displayed above each of your channel. The 4 nearest channels are pre-selected by default for each empty template channels, but you can modify these selections as needed. If you uncheck all the checkboxes for a particular template channel, it will be filled with zeros.
30
  - **Zero** method: All empty template channels are filled with zeros.
31
- Choose the method that best suits your needs, considering that the model's performance may vary depending on the method used.
32
-
33
- ### Step4: Auto-mapping Remaining Channels
34
- After completing the initial mapping steps, any channels that are not yet assigned to a template will be processed in this step. These remaining channels will be automatically mapped in batches, with a batch size of up to 30 channels. If the final batch contains fewer than 30 channels, the **Mean** method from Step3 will be applied to fill the remaining template channels.
35
-
36
 
37
- ### Mapping Result
 
 
38
 
39
  ## 2. Decode data
40
- In this phase, you can select which model to use for denoising your EEG data. Detailed information about the models can be found in the other tabs.
41
- 1. **Initial model run**: Using the results from the previous mapping steps, the model will first run on the processed data. After this initial run, the reconstructed signals will be restored to their original positions in your data.
42
- 2. **Subsequent processing**: If your data contains more than 30 channels, after the initial model run, we will automatically handle any remaining channels that were not initially processed. These channels will be mapped to the template channels using the Hungarian Algorithm, which optimally assigns each input channel to a template channel based on proximity. The reconstructed signals will then be placed back into their original positions in your data. This process continues iteratively until all input channels are reconstructed. If the total number of your channels is not a multiple of 30, the final iteration may result in some template channels being empty. In this case, **Mean** filling method from Step3 will be used to fill the values of these remaining empty template channels.
43
-
44
  """
45
 
46
  icunet = """
@@ -222,75 +225,75 @@ with gr.Blocks() as demo:
222
  with gr.Row():
223
  gr.Markdown(
224
  """
225
- <p style="text-align: center;">(...)</p>
226
  """
227
  )
228
  with gr.Row():
229
 
230
  with gr.Column():
231
- gr.Markdown("# 1.Channel Mapping")
232
- # ------------------------input--------------------------
233
- gr.Markdown("""
234
- - The data need to be a two-dimentional array (channel, timepoint).
235
- - If you don't have the channel location file, we recommend you to download
236
- the standard montage <a href="">here</a>. If the channels in those files
237
- don't match yours, you can use **EEGLAB** to modify them to your needed montage.
238
- """)
239
- with gr.Row():
240
- in_data_file = gr.File(label="Raw data (.csv)", file_types=[".csv"])
241
- in_loc_file = gr.File(label="Channel locations (.loc, .locs, .xyz, .sfp, .txt)",
242
- file_types=[".loc", "locs", ".xyz", ".sfp", ".txt"])
243
- with gr.Row():
244
- in_samplerate = gr.Textbox(label="Sampling rate (Hz)", scale=2)
245
- map_btn = gr.Button("Mapping", interactive=False, scale=1)
246
-
247
- # ------------------------mapping------------------------
248
- desc_md = gr.Markdown(visible=False)
249
- # step1 : initial mapping abd rescaling
250
- with gr.Row():
251
- tpl_img = gr.Image("./template_montage.png", label="Template channels", visible=False)
252
- mapped_img = gr.Image(label="Input channels", visible=False)
253
- # step2 : forward unmatched input channels to empty template channels
254
- radio_group = gr.Radio(elem_id="radio-group", visible=False)
255
- # step3 : fill the remaining template channels
256
- with gr.Row():
257
- in_fillmode = gr.Dropdown(choices=["mean", "zero"],
258
- value="mean",
259
- label="Filling method",
260
- visible=False,
261
- scale=2)
262
- fillmode_btn = gr.Button("OK", visible=False, scale=1)
263
- chkbox_group = gr.CheckboxGroup(elem_id="chkbox-group", visible=False)
264
- # step4 : mapping result
265
- out_json_file = gr.File(label="Mapping result", visible=False)
266
-
267
- with gr.Row():
268
- clear_btn = gr.Button("Clear", visible=False)
269
- step2_btn = gr.Button("Next", visible=False)
270
- step3_btn = gr.Button("Next", visible=False)
271
- next_btn = gr.Button("Next step", visible=False)
272
- # -------------------------------------------------------
273
 
274
  with gr.Column():
275
- gr.Markdown("# 2.Decode Data")
276
- # ------------------------input--------------------------
277
- with gr.Row():
278
- in_modelname = gr.Dropdown(choices=[
279
- ("ART", "EEGART"),
280
- ("IC-U-Net", "ICUNet"),
281
- ("IC-U-Net++", "UNetpp"),
282
- ("IC-U-Net-Attn", "AttUnet"),
283
- "(mapped data)",
284
- "(denoised data)"],
285
- value="EEGART",
286
- label="Model",
287
- scale=2)
288
- run_btn = gr.Button(scale=1, interactive=False)
289
-
290
- # ------------------------output-------------------------
291
- batch_md = gr.Markdown(visible=False)
292
- out_data_file = gr.File(label="Denoised data", visible=False)
293
- # -------------------------------------------------------
 
 
 
294
 
295
  with gr.Row():
296
  with gr.Tab("ART"):
@@ -304,31 +307,20 @@ with gr.Blocks() as demo:
304
  with gr.Tab("README"):
305
  gr.Markdown(readme)
306
 
307
- #demo.load(js=js)
308
-
309
- # verify that all required inputs have been provided
310
- @gr.on(triggers = [in_data_file.upload, in_data_file.clear, in_loc_file.upload, in_loc_file.clear, in_samplerate.change],
311
- inputs = [in_data_file, in_loc_file, in_samplerate], outputs = map_btn)
312
- def check_input(in_data, in_loc, samplerate):
313
- if in_data!=None and in_loc!=None and samplerate!="":
314
- return gr.Button(interactive=True)
315
- else:
316
- return gr.Button(interactive=False)
317
-
318
 
319
  # +========================================================================================+
320
  # | Stage1: channel mapping |
321
  # +========================================================================================+
322
- def reset_all(in_data, in_loc, samplerate):
323
  # establish a new folder for the current session
324
- rootpath = os.path.dirname(str(in_data))
325
  try:
326
  os.mkdir(rootpath+"/session_data/")
327
  except OSError as e:
328
  utils.dataDelete(rootpath+"/session_data/")
329
  os.mkdir(rootpath+"/session_data/")
330
  print(e)
331
- # establish new folders for stage1 and stage2
332
  os.mkdir(rootpath+"/session_data/stage1/")
333
  os.mkdir(rootpath+"/session_data/stage2/")
334
 
@@ -340,7 +332,6 @@ with gr.Blocks() as demo:
340
  "stage1" : {
341
  "filePath" : rootpath+"/session_data/stage1/",
342
  "fileNames" : {
343
- "input_data" : in_data,
344
  "input_loc" : in_loc,
345
  "input_montage" : "",
346
  "mapped_montage" : ""
@@ -361,12 +352,12 @@ with gr.Blocks() as demo:
361
  "stage2" : {
362
  "filePath" : rootpath+"/session_data/stage2/",
363
  "fileNames" : {
 
364
  "output_data" : ""
365
  },
366
  "totalBatchNum" : None
367
  }
368
  }
369
- # reset layout
370
  return {app_info_json : app_info,
371
  channel_info_json : channel_info,
372
  # --------------------Stage1-------------------------
@@ -383,7 +374,9 @@ with gr.Blocks() as demo:
383
  chkbox_group : gr.CheckboxGroup(choices=[], value=[], label="", visible=False),
384
  step3_btn : gr.Button(visible=False),
385
  out_json_file : gr.File(value=None, visible=False),
 
386
  # --------------------Stage2-------------------------
 
387
  run_btn : gr.Button(interactive=False),
388
  batch_md : gr.Markdown(visible=False),
389
  out_data_file : gr.File(visible=False)}
@@ -400,17 +393,13 @@ with gr.Blocks() as demo:
400
  # ========================================step0=========================================
401
  # step0 to step1
402
  if stage1_info["state"] == "step1-initializing":
403
- #print('step0 -> step1')
404
-
405
- # 1. match the names of in_channels and tpl_channels
406
  yield {desc_md : gr.Markdown("Mapping...", visible=True)}
 
 
407
  stage1_info, channel_info, tpl_montage, in_montage = app_utils.match_names(stage1_info, channel_info)
408
-
409
- # 2. rescale coordinates
410
- yield {desc_md : gr.Markdown("Rescaling...")}
411
  channel_info = app_utils.align_coords(channel_info, tpl_montage, in_montage)
412
-
413
- # 3. generate and save figures of the montages
414
  filename1 = filepath+"input_montage_"+str(random.randint(1,10000))+".png"
415
  filename2 = filepath+"mapped_montage_"+str(random.randint(1,10000))+".png"
416
  channel_info = app_utils.save_figures(channel_info, tpl_montage, filename1, filename2)
@@ -418,24 +407,19 @@ with gr.Blocks() as demo:
418
  "input_montage" : filename1,
419
  "mapped_montage" : filename2
420
  })
421
-
422
- # 4. matching result
423
  # check if there are red dots (unmatched in_channels) on the input montage
424
  unassigned_num = len(stage1_info["unassignedInputs"])
425
  if unassigned_num == 0:
426
  md = """
427
- ---
428
  ### Step1: Initial Matching and Rescaling
429
  Below is the result of mapping your channels to our template channels based on their names.
430
  """
431
  else:
432
  md = """
433
- ---
434
  ### Step1: Initial Matching and Rescaling
435
  Below is the result of mapping your channels to our template channels based on their names.
436
  - channels highlighted in red are those that do not match any template channels.
437
  """
438
-
439
  stage1_info["state"] = "step1-finished"
440
  app_info["stage1"] = stage1_info
441
  yield {app_info_json : app_info,
@@ -455,13 +439,6 @@ with gr.Blocks() as demo:
455
  # step1 to step4
456
  # the in_channels has all the 30 tpl_channels (in_num>=30)
457
  if matched_num == 30:
458
- #print('step1 -> step4')
459
- md = """
460
- ---
461
- ### Mapping result
462
- (...)
463
- """
464
-
465
  # finalize and save the mapping results
466
  filename = filepath+"mapping_result.json"
467
  stage1_info, stage2_info, channel_info = app_utils.mapping_result(
@@ -472,24 +449,22 @@ with gr.Blocks() as demo:
472
  app_info["stage2"] = stage2_info
473
  yield {app_info_json : app_info,
474
  channel_info_json : channel_info,
475
- desc_md : gr.Markdown(md),
476
  tpl_img : gr.Image(visible=False),
477
  mapped_img : gr.Image(visible=False),
478
  next_btn : gr.Button(visible=False),
479
  out_json_file : gr.File(filename, visible=True),
 
480
  run_btn : gr.Button(interactive=True)}
481
 
482
  # step1 to step2
483
  # matched_num < 30, and there're still some unmatched in_channels
484
  elif in_num > matched_num:
485
- #print('step1 -> step2')
486
  md = """
487
- ---
488
  ### Step2: Forwarding Unmatched Channels
489
  Select one of your unmatched channels to forward its data to the empty template channel
490
  currently indicated in red.
491
  """
492
-
493
  # initialize the progress indication label for step2
494
  stage1_info.update({
495
  "fillingCount" : 1,
@@ -521,15 +496,12 @@ with gr.Blocks() as demo:
521
  # step1 to step3-1
522
  # in_num < 30, but all of them can match to some tpl_channels
523
  elif in_num == matched_num:
524
- #print('step1 -> step3-1')
525
  md = """
526
- ---
527
  ### Step3: Filling Remaining Template Channels
528
  To run the model successfully, we need to ensure that all 30 template channels are filled.
529
  In this step, you are required to select one of the methods provided below to fill the
530
  remaining empty template channels.
531
  """
532
-
533
  stage1_info["state"] = "step3-select-method"
534
  app_info["stage1"] = stage1_info
535
  yield {app_info_json : app_info,
@@ -570,13 +542,6 @@ with gr.Blocks() as demo:
570
  # step2 to step4
571
  # all the unmatched tpl_channels were filled by in_channels
572
  if len(stage1_info["missingTemplates"]) == 0:
573
- #print('step2 -> step4')
574
- md = """
575
- ---
576
- ### Mapping result
577
- (...)
578
- """
579
-
580
  # finalize and save the mapping results
581
  filename = filepath+"mapping_result.json"
582
  stage1_info, stage2_info, channel_info = app_utils.mapping_result(
@@ -587,23 +552,21 @@ with gr.Blocks() as demo:
587
  app_info["stage2"] = stage2_info
588
  yield {app_info_json : app_info,
589
  channel_info_json : channel_info,
590
- desc_md : gr.Markdown(md),
591
  radio_group : gr.Radio(visible=False),
592
  out_json_file : gr.File(filename, visible=True),
 
593
  clear_btn : gr.Button(visible=False),
594
  next_btn : gr.Button(visible=False),
595
  run_btn : gr.Button(interactive=True)}
596
  # step2 to step3-1
597
  else:
598
- #print('step2 -> step3-1')
599
  md = """
600
- ---
601
  ### Step3: Filling Remaining Template Channels
602
  To run the model successfully, we need to ensure that all 30 template channels are filled.
603
  In this step, you are required to select one of the methods provided below to fill the
604
  remaining empty template channels.
605
  """
606
-
607
  stage1_info["state"] = "step3-select-method"
608
  app_info["stage1"] = stage1_info
609
  yield {app_info_json : app_info,
@@ -620,13 +583,6 @@ with gr.Blocks() as demo:
620
 
621
  # step3-1 to step4
622
  if fillmode == "zero":
623
- #print('step3-1 -> step4')
624
- md = """
625
- ---
626
- ### Mapping result
627
- (...)
628
- """
629
-
630
  # finalize and save the mapping results
631
  filename = filepath+"mapping_result.json"
632
  stage1_info, stage2_info, channel_info = app_utils.mapping_result(
@@ -637,27 +593,24 @@ with gr.Blocks() as demo:
637
  app_info["stage2"] = stage2_info
638
  yield {app_info_json : app_info,
639
  channel_info_json : channel_info,
640
- desc_md : gr.Markdown(md),
641
  in_fillmode : gr.Dropdown(visible=False),
642
  fillmode_btn : gr.Button(visible=False),
643
  out_json_file : gr.File(filename, visible=True),
 
644
  run_btn : gr.Button(interactive=True)}
645
  # step3-1 to step3-2
646
  elif fillmode == "mean":
647
- #print('step3-1 -> step3-2')
648
  md = """
649
- ---
650
  ### Step3: Fill the remaining template channels
651
  The current empty template channel, indicated in red, will be filled with the average
652
  value of the data from the selected channels. (By default, the 4 nearest channels are pre-selected.)
653
  """
654
-
655
  # find the 4 nearest in_channels for each unmatched tpl_channels
656
  stage1_info["mappingData"][0]["newOrder"] = app_utils.find_neighbors(
657
  channel_info,
658
  stage1_info["missingTemplates"],
659
  stage1_info["mappingData"][0]["newOrder"])
660
-
661
  # initialize the progress indication label
662
  stage1_info.update({
663
  "fillingCount" : 1,
@@ -692,7 +645,6 @@ with gr.Blocks() as demo:
692
  # =======================================step3-2========================================
693
  # step3-2 to step4
694
  elif stage1_info["state"] == "step3-2-selecting":
695
- #print('step3-2 -> step4')
696
 
697
  # --------------------store information before the button click---------------------
698
  # if the user didn't uncheck all in_channel checkboxes
@@ -705,12 +657,6 @@ with gr.Blocks() as demo:
705
  stage1_info["mappingData"][0]["newOrder"][prev_target_idx] = selected_indices
706
  #print(f'{prev_target_name}({prev_target_idx}): {selected_indices}')
707
  # ----------------------------------------------------------------------------------
708
- md = """
709
- ---
710
- ### Mapping result
711
- (...)
712
- """
713
-
714
  # finalize and save the mapping results
715
  filename = filepath+"mapping_result.json"
716
  stage1_info, stage2_info, channel_info = app_utils.mapping_result(
@@ -721,17 +667,18 @@ with gr.Blocks() as demo:
721
  app_info["stage2"] = stage2_info
722
  yield {app_info_json : app_info,
723
  channel_info_json : channel_info,
724
- desc_md : gr.Markdown(md),
725
  chkbox_group : gr.CheckboxGroup(visible=False),
726
  next_btn : gr.Button(visible=False),
727
  out_json_file : gr.File(filename, visible=True),
 
728
  run_btn : gr.Button(interactive=True)}
729
 
730
  next_btn.click(
731
  fn = init_next_step,
732
  inputs = [app_info_json, channel_info_json, in_fillmode, radio_group, chkbox_group],
733
  outputs = [app_info_json, channel_info_json, desc_md, tpl_img, mapped_img, radio_group, clear_btn, step2_btn,
734
- in_fillmode, fillmode_btn, chkbox_group, step3_btn, out_json_file, next_btn, run_btn]
735
  ).success(
736
  fn = None,
737
  js = init_js,
@@ -745,10 +692,10 @@ with gr.Blocks() as demo:
745
  # +========================================================================================+
746
  map_btn.click(
747
  fn = reset_all,
748
- inputs = [in_data_file, in_loc_file, in_samplerate],
749
  outputs = [app_info_json, channel_info_json, map_btn, desc_md, next_btn, tpl_img, mapped_img,
750
- radio_group, clear_btn, step2_btn, in_fillmode, fillmode_btn, chkbox_group, step3_btn, out_json_file,
751
- run_btn, batch_md, out_data_file]
752
  ).success(
753
  fn = init_next_step,
754
  inputs = [app_info_json, channel_info_json, in_fillmode, radio_group, chkbox_group],
@@ -781,7 +728,6 @@ with gr.Blocks() as demo:
781
 
782
  def update_radio(app_info, channel_info, selected):
783
  stage1_info = app_info["stage1"]
784
-
785
  # ----------------------store information before the button click-----------------------
786
  # check if the user has selected an in_channel to forward to the previous target tpl_channel
787
  if selected != []:
@@ -838,7 +784,6 @@ with gr.Blocks() as demo:
838
  # +========================================================================================+
839
  def update_chkbox(app_info, channel_info, selected):
840
  stage1_info = app_info["stage1"]
841
-
842
  # ----------------------store information before the button click-----------------------
843
  # if the user didn't uncheck all in_channel checkboxes
844
  if selected != []:
@@ -875,7 +820,7 @@ with gr.Blocks() as demo:
875
  fn = init_next_step,
876
  inputs = [app_info_json, channel_info_json, in_fillmode, radio_group, chkbox_group],
877
  outputs = [app_info_json, channel_info_json, desc_md, in_fillmode, fillmode_btn, chkbox_group, step3_btn,
878
- out_json_file, next_btn, run_btn]
879
  ).success(
880
  fn = None,
881
  js = init_js,
@@ -898,7 +843,7 @@ with gr.Blocks() as demo:
898
  # +========================================================================================+
899
  # | Stage2: decode data |
900
  # +========================================================================================+
901
- def reset_run(app_info, modelname):
902
  stage1_info = app_info["stage1"]
903
  stage2_info = app_info["stage2"]
904
 
@@ -909,13 +854,13 @@ with gr.Blocks() as demo:
909
  new_filepath = app_info["rootPath"]+"stage2_"+str(random.randint(1,10000))+"/"
910
  os.mkdir(new_filepath)
911
  # generate the output filename
912
- filename = stage1_info["fileNames"]["input_data"]
913
- filename = os.path.basename(str(filename))
914
  new_filename = os.path.splitext(filename)[0]+'_'+modelname+'.csv'
915
 
916
  stage2_info.update({
917
  "filePath" : new_filepath,
918
  "fileNames" : {
 
919
  "output_data" : new_filepath + new_filename
920
  }
921
  })
@@ -931,13 +876,10 @@ with gr.Blocks() as demo:
931
 
932
  filepath = stage2_info["filePath"]
933
  samplerate = app_info["sampleRate"]
934
- filename = stage1_info["fileNames"]["input_data"]
935
  new_filename = stage2_info["fileNames"]["output_data"]
936
 
937
- # flag to indicate if the process has been interrupted by the user
938
- break_flag = False
939
-
940
- # run the model multiple times until all in_channels are reconstructed
941
  for i in range(stage2_info["totalBatchNum"]):
942
  # establish a temp folder
943
  try:
@@ -992,7 +934,7 @@ with gr.Blocks() as demo:
992
 
993
  run_btn.click(
994
  fn = reset_run,
995
- inputs = [app_info_json, in_modelname],
996
  outputs = [app_info_json, run_btn, batch_md, out_data_file]
997
 
998
  ).success(
 
6
 
7
  readme = """
8
 
9
+ This tool serves two main purposes:
10
+ 1. **Channel Mapping**: Align your EEG channels with our template channels to ensure compatibility with our models.
11
+ 2. **EEG Data Denoising**: Use our pre-trained models—**ART**, **IC-U-Net**, **IC-U-Net++**, and **IC-U-Net-Attn**—to denoise your EEG data.
12
+
13
+ ## File Requirements and Preparation
14
+ - **Channel locations**: If you don't have the channel location file, we recommend you to download the standard montage <a href="">here</a>. If the channels in those files don't match yours, you can use **EEGLAB** to adjust them to your required montage.
15
+ - **Raw data**: The data need to be a two-dimentional array (channel, timepoint).
16
+ - **Channel requirements**: Your data must include some channels that correspond to our template channels, including: ``Fp1, Fp2, F7, F3, Fz, F4, F8, FT7, FC3, FCz, FC4, FT8, T7, C3, Cz, C4, T8, TP7, CP3, CPz, CP4, TP8, P7, P3, Pz, P4, P8, O1, Oz, O2``. At least some of them need to be present for successful mapping.
17
+ - **Channel removal**: Before uploading your files, please remove any reference, ECG, EOG, EMG... channels.
18
 
19
  ## 1. Channel Mapping
20
+ The following steps will guide you through the process of mapping your EEG channels to our template channels.
21
 
22
  ### Step1: Initial Matching and Rescaling
23
  After clicking on ``Mapping`` button, we will first match your channels to our template channels by their names. Using the matched channels as reference points, we will apply Thin Plate Spline (TPS) transformation to rescale your montage to align with our template's scale. The template montage and your rescaled montage will be displayed side by side for comparison. Channels that do not have a match in our template will be **highlighted in red**.
24
+ - If your data includes all the 30 template channels, you will be directed to **Mapping Results**.
25
  - If your data doesn't include all the 30 template channels and you have some channels that do not match the template, you will be directed to **Step2**.
26
  - If all your channels are included in our template but you have fewer than 30 channels, you will be directed to **Step3**.
27
 
28
  ### Step2: Forwarding Unmatched Channels
29
  In this step, you will handle the channels that didn't have a direct match with our template, by manually assigning them to the template channels that are still empty, ensuring the most efficient use of your data.
30
  Your unmatched channels, previously highlighted in red, will be shown on your montage with a radio button displayed above each. You can choose to forward the data from these unmatched channels to the empty template channels. The interface will display each empty template channel in sequence, allowing you to select which of your unmatched channels to forward.
31
+ - If all empty template channels are filled by your selections, you will be directed to **Mapping Results**.
32
  - If there are still empty template channels remaining, you will be directed to **Step3**.
33
 
34
  ### Step3: Filling Remaining Template Channels
35
  To run the models successfully, we need to ensure that all 30 template channels are filled. In this step, you are required to select one of the methods provided below to fill the remaining empty template channels:
36
  - **Mean** method: Each empty template channel is filled with the average value of data from the nearest input channels. By default, the 4 closest input channels (determined after aligning your montage to the template's scale using TPS) are selected for this averaging process. On the interface, you will see checkboxes displayed above each of your channel. The 4 nearest channels are pre-selected by default for each empty template channels, but you can modify these selections as needed. If you uncheck all the checkboxes for a particular template channel, it will be filled with zeros.
37
  - **Zero** method: All empty template channels are filled with zeros.
38
+ Choose the method that best suits your needs, considering that the model's performance may vary depending on the method used.
39
+ Once all template channels are filled, you will be directed to **Mapping Results**.
 
 
 
40
 
41
+ ### Mapping Results
42
+ After completing the previous steps, your channels will be aligned with the template channels required by our models. In case there are still some channels that haven't been mapped, we will automatically batch and optimally assign them to the template.
43
+ Once the channel mapping process is completed, a **JSON file** containing the mapping results will be generated. This file is necessary only if you plan to run the models using the source code; otherwise, you can ignore it.
44
 
45
  ## 2. Decode data
46
+ After clicking the ``Run`` button, we will process your EEG data based on the mapping results. If necessary, your data will be devided into batches and run the models on each batch sequentially, ensuring that all channels are properly processed.
 
 
 
47
  """
48
 
49
  icunet = """
 
225
  with gr.Row():
226
  gr.Markdown(
227
  """
228
+ <p style="text-align: center;">...</p>
229
  """
230
  )
231
  with gr.Row():
232
 
233
  with gr.Column():
234
+ with gr.Row(variant='panel'):
235
+ with gr.Column():
236
+ gr.Markdown("# 1.Channel Mapping")
237
+ # ------------------------input--------------------------
238
+ in_loc_file = gr.File(label="Channel locations (.loc, .locs, .xyz, .sfp, .txt)",
239
+ file_types=[".loc", "locs", ".xyz", ".sfp", ".txt"])
240
+ with gr.Row():
241
+ in_samplerate = gr.Textbox(label="Sampling rate (Hz)", scale=2)
242
+ map_btn = gr.Button("Mapping", scale=1)
243
+
244
+ # ------------------------mapping------------------------
245
+ desc_md = gr.Markdown(visible=False)
246
+ # step1 : initial matching and rescaling
247
+ with gr.Row():
248
+ tpl_img = gr.Image("./template_montage.png", label="Template channels", visible=False)
249
+ mapped_img = gr.Image(label="Input channels", visible=False)
250
+ # step2 : forward unmatched input channels to empty template channels
251
+ radio_group = gr.Radio(elem_id="radio-group", visible=False)
252
+ # step3 : fill the remaining template channels
253
+ with gr.Row():
254
+ in_fillmode = gr.Dropdown(choices=["mean", "zero"],
255
+ value="mean",
256
+ label="Filling method",
257
+ visible=False,
258
+ scale=2)
259
+ fillmode_btn = gr.Button("OK", visible=False, scale=1)
260
+ chkbox_group = gr.CheckboxGroup(elem_id="chkbox-group", visible=False)
261
+ # step4 : mapping result
262
+ out_json_file = gr.File(label="Mapping result", visible=False)
263
+ res_md = gr.Markdown("""
264
+ (Download this file if you plan to run the models using the source code.)
265
+ """, visible=False)
266
+
267
+ with gr.Row():
268
+ clear_btn = gr.Button("Clear", visible=False)
269
+ step2_btn = gr.Button("Next", visible=False)
270
+ step3_btn = gr.Button("Next", visible=False)
271
+ next_btn = gr.Button("Next step", visible=False)
272
+ # -------------------------------------------------------
 
 
 
273
 
274
  with gr.Column():
275
+ with gr.Row(variant='panel'):
276
+ with gr.Column():
277
+ gr.Markdown("# 2.Decode Data")
278
+ # ------------------------input--------------------------
279
+ in_data_file = gr.File(label="Raw data (.csv)", file_types=[".csv"])
280
+ with gr.Row():
281
+ in_modelname = gr.Dropdown(choices=[
282
+ ("ART", "EEGART"),
283
+ ("IC-U-Net", "ICUNet"),
284
+ ("IC-U-Net++", "UNetpp"),
285
+ ("IC-U-Net-Attn", "AttUnet")],
286
+ #"(mapped data)",
287
+ #"(denoised data)"],
288
+ value="EEGART",
289
+ label="Model",
290
+ scale=2)
291
+ run_btn = gr.Button(interactive=False, scale=1)
292
+
293
+ # ------------------------output-------------------------
294
+ batch_md = gr.Markdown(visible=False)
295
+ out_data_file = gr.File(label="Denoised data", visible=False)
296
+ # -------------------------------------------------------
297
 
298
  with gr.Row():
299
  with gr.Tab("ART"):
 
307
  with gr.Tab("README"):
308
  gr.Markdown(readme)
309
 
310
+ #demo.load(js=js)
 
 
 
 
 
 
 
 
 
 
311
 
312
  # +========================================================================================+
313
  # | Stage1: channel mapping |
314
  # +========================================================================================+
315
+ def reset_all(in_loc, samplerate):
316
  # establish a new folder for the current session
317
+ rootpath = os.path.dirname(str(in_loc))
318
  try:
319
  os.mkdir(rootpath+"/session_data/")
320
  except OSError as e:
321
  utils.dataDelete(rootpath+"/session_data/")
322
  os.mkdir(rootpath+"/session_data/")
323
  print(e)
 
324
  os.mkdir(rootpath+"/session_data/stage1/")
325
  os.mkdir(rootpath+"/session_data/stage2/")
326
 
 
332
  "stage1" : {
333
  "filePath" : rootpath+"/session_data/stage1/",
334
  "fileNames" : {
 
335
  "input_loc" : in_loc,
336
  "input_montage" : "",
337
  "mapped_montage" : ""
 
352
  "stage2" : {
353
  "filePath" : rootpath+"/session_data/stage2/",
354
  "fileNames" : {
355
+ "input_data" : "",
356
  "output_data" : ""
357
  },
358
  "totalBatchNum" : None
359
  }
360
  }
 
361
  return {app_info_json : app_info,
362
  channel_info_json : channel_info,
363
  # --------------------Stage1-------------------------
 
374
  chkbox_group : gr.CheckboxGroup(choices=[], value=[], label="", visible=False),
375
  step3_btn : gr.Button(visible=False),
376
  out_json_file : gr.File(value=None, visible=False),
377
+ res_md : gr.Markdown(visible=False),
378
  # --------------------Stage2-------------------------
379
+ in_data_file : gr.File(value=None),
380
  run_btn : gr.Button(interactive=False),
381
  batch_md : gr.Markdown(visible=False),
382
  out_data_file : gr.File(visible=False)}
 
393
  # ========================================step0=========================================
394
  # step0 to step1
395
  if stage1_info["state"] == "step1-initializing":
 
 
 
396
  yield {desc_md : gr.Markdown("Mapping...", visible=True)}
397
+
398
+ # match the names of in_channels and tpl_channels
399
  stage1_info, channel_info, tpl_montage, in_montage = app_utils.match_names(stage1_info, channel_info)
400
+ # rescale coordinates
 
 
401
  channel_info = app_utils.align_coords(channel_info, tpl_montage, in_montage)
402
+ # generate and save figures of the montages
 
403
  filename1 = filepath+"input_montage_"+str(random.randint(1,10000))+".png"
404
  filename2 = filepath+"mapped_montage_"+str(random.randint(1,10000))+".png"
405
  channel_info = app_utils.save_figures(channel_info, tpl_montage, filename1, filename2)
 
407
  "input_montage" : filename1,
408
  "mapped_montage" : filename2
409
  })
 
 
410
  # check if there are red dots (unmatched in_channels) on the input montage
411
  unassigned_num = len(stage1_info["unassignedInputs"])
412
  if unassigned_num == 0:
413
  md = """
 
414
  ### Step1: Initial Matching and Rescaling
415
  Below is the result of mapping your channels to our template channels based on their names.
416
  """
417
  else:
418
  md = """
 
419
  ### Step1: Initial Matching and Rescaling
420
  Below is the result of mapping your channels to our template channels based on their names.
421
  - channels highlighted in red are those that do not match any template channels.
422
  """
 
423
  stage1_info["state"] = "step1-finished"
424
  app_info["stage1"] = stage1_info
425
  yield {app_info_json : app_info,
 
439
  # step1 to step4
440
  # the in_channels has all the 30 tpl_channels (in_num>=30)
441
  if matched_num == 30:
 
 
 
 
 
 
 
442
  # finalize and save the mapping results
443
  filename = filepath+"mapping_result.json"
444
  stage1_info, stage2_info, channel_info = app_utils.mapping_result(
 
449
  app_info["stage2"] = stage2_info
450
  yield {app_info_json : app_info,
451
  channel_info_json : channel_info,
452
+ desc_md : gr.Markdown(visible=False),
453
  tpl_img : gr.Image(visible=False),
454
  mapped_img : gr.Image(visible=False),
455
  next_btn : gr.Button(visible=False),
456
  out_json_file : gr.File(filename, visible=True),
457
+ res_md : gr.Markdown(visible=True),
458
  run_btn : gr.Button(interactive=True)}
459
 
460
  # step1 to step2
461
  # matched_num < 30, and there're still some unmatched in_channels
462
  elif in_num > matched_num:
 
463
  md = """
 
464
  ### Step2: Forwarding Unmatched Channels
465
  Select one of your unmatched channels to forward its data to the empty template channel
466
  currently indicated in red.
467
  """
 
468
  # initialize the progress indication label for step2
469
  stage1_info.update({
470
  "fillingCount" : 1,
 
496
  # step1 to step3-1
497
  # in_num < 30, but all of them can match to some tpl_channels
498
  elif in_num == matched_num:
 
499
  md = """
 
500
  ### Step3: Filling Remaining Template Channels
501
  To run the model successfully, we need to ensure that all 30 template channels are filled.
502
  In this step, you are required to select one of the methods provided below to fill the
503
  remaining empty template channels.
504
  """
 
505
  stage1_info["state"] = "step3-select-method"
506
  app_info["stage1"] = stage1_info
507
  yield {app_info_json : app_info,
 
542
  # step2 to step4
543
  # all the unmatched tpl_channels were filled by in_channels
544
  if len(stage1_info["missingTemplates"]) == 0:
 
 
 
 
 
 
 
545
  # finalize and save the mapping results
546
  filename = filepath+"mapping_result.json"
547
  stage1_info, stage2_info, channel_info = app_utils.mapping_result(
 
552
  app_info["stage2"] = stage2_info
553
  yield {app_info_json : app_info,
554
  channel_info_json : channel_info,
555
+ desc_md : gr.Markdown(visible=False),
556
  radio_group : gr.Radio(visible=False),
557
  out_json_file : gr.File(filename, visible=True),
558
+ res_md : gr.Markdown(visible=True),
559
  clear_btn : gr.Button(visible=False),
560
  next_btn : gr.Button(visible=False),
561
  run_btn : gr.Button(interactive=True)}
562
  # step2 to step3-1
563
  else:
 
564
  md = """
 
565
  ### Step3: Filling Remaining Template Channels
566
  To run the model successfully, we need to ensure that all 30 template channels are filled.
567
  In this step, you are required to select one of the methods provided below to fill the
568
  remaining empty template channels.
569
  """
 
570
  stage1_info["state"] = "step3-select-method"
571
  app_info["stage1"] = stage1_info
572
  yield {app_info_json : app_info,
 
583
 
584
  # step3-1 to step4
585
  if fillmode == "zero":
 
 
 
 
 
 
 
586
  # finalize and save the mapping results
587
  filename = filepath+"mapping_result.json"
588
  stage1_info, stage2_info, channel_info = app_utils.mapping_result(
 
593
  app_info["stage2"] = stage2_info
594
  yield {app_info_json : app_info,
595
  channel_info_json : channel_info,
596
+ desc_md : gr.Markdown(visible=False),
597
  in_fillmode : gr.Dropdown(visible=False),
598
  fillmode_btn : gr.Button(visible=False),
599
  out_json_file : gr.File(filename, visible=True),
600
+ res_md : gr.Markdown(visible=True),
601
  run_btn : gr.Button(interactive=True)}
602
  # step3-1 to step3-2
603
  elif fillmode == "mean":
 
604
  md = """
 
605
  ### Step3: Fill the remaining template channels
606
  The current empty template channel, indicated in red, will be filled with the average
607
  value of the data from the selected channels. (By default, the 4 nearest channels are pre-selected.)
608
  """
 
609
  # find the 4 nearest in_channels for each unmatched tpl_channels
610
  stage1_info["mappingData"][0]["newOrder"] = app_utils.find_neighbors(
611
  channel_info,
612
  stage1_info["missingTemplates"],
613
  stage1_info["mappingData"][0]["newOrder"])
 
614
  # initialize the progress indication label
615
  stage1_info.update({
616
  "fillingCount" : 1,
 
645
  # =======================================step3-2========================================
646
  # step3-2 to step4
647
  elif stage1_info["state"] == "step3-2-selecting":
 
648
 
649
  # --------------------store information before the button click---------------------
650
  # if the user didn't uncheck all in_channel checkboxes
 
657
  stage1_info["mappingData"][0]["newOrder"][prev_target_idx] = selected_indices
658
  #print(f'{prev_target_name}({prev_target_idx}): {selected_indices}')
659
  # ----------------------------------------------------------------------------------
 
 
 
 
 
 
660
  # finalize and save the mapping results
661
  filename = filepath+"mapping_result.json"
662
  stage1_info, stage2_info, channel_info = app_utils.mapping_result(
 
667
  app_info["stage2"] = stage2_info
668
  yield {app_info_json : app_info,
669
  channel_info_json : channel_info,
670
+ desc_md : gr.Markdown(visible=False),
671
  chkbox_group : gr.CheckboxGroup(visible=False),
672
  next_btn : gr.Button(visible=False),
673
  out_json_file : gr.File(filename, visible=True),
674
+ res_md : gr.Markdown(visible=True),
675
  run_btn : gr.Button(interactive=True)}
676
 
677
  next_btn.click(
678
  fn = init_next_step,
679
  inputs = [app_info_json, channel_info_json, in_fillmode, radio_group, chkbox_group],
680
  outputs = [app_info_json, channel_info_json, desc_md, tpl_img, mapped_img, radio_group, clear_btn, step2_btn,
681
+ in_fillmode, fillmode_btn, chkbox_group, step3_btn, out_json_file, res_md, next_btn, run_btn]
682
  ).success(
683
  fn = None,
684
  js = init_js,
 
692
  # +========================================================================================+
693
  map_btn.click(
694
  fn = reset_all,
695
+ inputs = [in_loc_file, in_samplerate],
696
  outputs = [app_info_json, channel_info_json, map_btn, desc_md, next_btn, tpl_img, mapped_img,
697
+ radio_group, clear_btn, step2_btn, in_fillmode, fillmode_btn, chkbox_group, step3_btn,
698
+ out_json_file, res_md, in_data_file, run_btn, batch_md, out_data_file]
699
  ).success(
700
  fn = init_next_step,
701
  inputs = [app_info_json, channel_info_json, in_fillmode, radio_group, chkbox_group],
 
728
 
729
  def update_radio(app_info, channel_info, selected):
730
  stage1_info = app_info["stage1"]
 
731
  # ----------------------store information before the button click-----------------------
732
  # check if the user has selected an in_channel to forward to the previous target tpl_channel
733
  if selected != []:
 
784
  # +========================================================================================+
785
  def update_chkbox(app_info, channel_info, selected):
786
  stage1_info = app_info["stage1"]
 
787
  # ----------------------store information before the button click-----------------------
788
  # if the user didn't uncheck all in_channel checkboxes
789
  if selected != []:
 
820
  fn = init_next_step,
821
  inputs = [app_info_json, channel_info_json, in_fillmode, radio_group, chkbox_group],
822
  outputs = [app_info_json, channel_info_json, desc_md, in_fillmode, fillmode_btn, chkbox_group, step3_btn,
823
+ out_json_file, res_md, next_btn, run_btn]
824
  ).success(
825
  fn = None,
826
  js = init_js,
 
843
  # +========================================================================================+
844
  # | Stage2: decode data |
845
  # +========================================================================================+
846
+ def reset_run(app_info, in_data, modelname):
847
  stage1_info = app_info["stage1"]
848
  stage2_info = app_info["stage2"]
849
 
 
854
  new_filepath = app_info["rootPath"]+"stage2_"+str(random.randint(1,10000))+"/"
855
  os.mkdir(new_filepath)
856
  # generate the output filename
857
+ filename = os.path.basename(str(in_data))
 
858
  new_filename = os.path.splitext(filename)[0]+'_'+modelname+'.csv'
859
 
860
  stage2_info.update({
861
  "filePath" : new_filepath,
862
  "fileNames" : {
863
+ "input_data" : in_data,
864
  "output_data" : new_filepath + new_filename
865
  }
866
  })
 
876
 
877
  filepath = stage2_info["filePath"]
878
  samplerate = app_info["sampleRate"]
879
+ filename = stage2_info["fileNames"]["input_data"]
880
  new_filename = stage2_info["fileNames"]["output_data"]
881
 
882
+ break_flag = False # flag to indicate if the process has been interrupted by the user
 
 
 
883
  for i in range(stage2_info["totalBatchNum"]):
884
  # establish a temp folder
885
  try:
 
934
 
935
  run_btn.click(
936
  fn = reset_run,
937
+ inputs = [app_info_json, in_data_file, in_modelname],
938
  outputs = [app_info_json, run_btn, batch_md, out_data_file]
939
 
940
  ).success(