MedTator

Loading data and initializing GUI ...

MedTator Settings

MEM: {{ get_metator_mem() }} MB
GUI
System
Other

Sentence Splitting Algorithm :

Save workspace in JSON file

Reset to Default Settings

Auto-save annotation (Experimental) :

Auto save/load configuration locally (Experimental) :

New UI for Toolkit (Experimental) :

New UI for Error Analysis (Development) :

Cohen's Kappa (Experimental) :

Linking marks selection between editor and concept list :

 

{{ msg_loading_anns }}
Loaded: {{ n_anns_loaded }} files

  • {{ v_ann._filename }}
    {{ anns[v_ann.idx].tags.length }}
/ {{ get_n_pages_by_total(virtual_anns.v_anns.length) }} Pages

Load schema file first
Drop a .yaml or .dtd File Here

Drop Annotation File(s) / Folder Here
(.xml)

  • All Tags
    {{ count_n_tags(null) }}
  • {{ etag.shortcut }}
    {{ count_n_tags(etag) }}
  • {{ rtag.shortcut }}
    {{ count_n_tags(rtag) }}
Tag ID Spans Text Attributes
{{ tag.tag }}
{{ tag.id }} {{ tag._annotator }} DOCUMENT {{ tag.spans }} LINK
{{ tag.text }}
{{ attr.name }} *  
Creating a Link Tag {{ linking_tag.tag }}
 - {{ attr.name }}

Annotation Schema

You need to prepare the annotation schema.

Annotation schema is needed to parse and format the given input files to XML files.
Please load the annotation schema file first in
the Annotation Tab

Schema: {{ dtd.name }}
- {{ dtd.etags.length }} Entity Tags
{{ tag.name }}
- {{ dtd.rtags.length }} Link Tags {{ tag.name }}

Raw Text Files

You need to prepare the .txt files.

Raw .txt Text Files

Loading files ...
Please wait for a little while.

Drag and drop the .txt files or the folder contains .txt files in this box.

Loaded {{ n_converter_corpus_raw_txt_files }} text files (.txt). As too many files are loaded, only show the first {{ max_converter_display_files }}.

{{ f.fn }}

MedTagger Files

You need to prepare matched .txt and .ann files.

Raw .txt Text Files

Loading files ...
Please wait for a little while.

Drag and drop the .txt files or the folder contains .txt files in this box

Loaded {{ n_converter_corpus_medtagger_txt_files }} text files (.txt). As too many files are loaded, only show the first {{ max_converter_display_files }}.

{{ f.fn }}
MedTagger .ann Files

Loading files ...
Please wait for a little while.

Drag and drop the .ann files or the folder contains .ann files in this box.

Loaded {{ n_converter_corpus_medtagger_ann_files }} MedTagger files (.ann). As too many files are loaded, only show the first {{ max_converter_display_files }}.

{{ f.lines.length }} {{ f.fn }}

Conversion Results

Click " Convert Files " to start conversion.

Generated MedTator XML Files

Converted {{ n_converter_results }} xml files. As too many files are converted, only show the first {{ max_converter_display_files }}.

{{ ann.tags.length }} {{ ann._filename }}

Export Preview

Tag Name
F1
OVERALL
 
{{ to_fixed(iaa_dict.all.f1) }}
{{ etag.name }}
 
{{ to_fixed(iaa_dict.tag[etag.name].f1) }}

Text Matched Files: {{ iaa_dict.stat.matched_hashcodes.length }}

Duplicated Files: {{ iaa_dict.stat.duplicates.length }}

Unmatched Files: {{ iaa_dict.stat.unmatched.length }}

Annotation Files

File names
OVERALL {{ iaa_display_tag_name }}   F1
A: {{ iaa_dict.ann[v_ann.ann_hashcode].anns[0]._filename }}
B: {{ iaa_dict.ann[v_ann.ann_hashcode].anns[1]._filename }}
GS: {{ iaa_gs_dict[v_ann.ann_hashcode].ann._filename }}
 
{{ to_fixed(get_rst(iaa_dict.ann[v_ann.ann_hashcode].rst)['f1']) }}
AB: {{ get_rst(iaa_dict.ann[v_ann.ann_hashcode].rst).cm.tp }} A+: {{ get_rst(iaa_dict.ann[v_ann.ann_hashcode].rst).cm.fp }} B+: {{ get_rst(iaa_dict.ann[v_ann.ann_hashcode].rst).cm.fn }}
OVERALL {{ iaa_display_tag_name }} in GS: {{ count_gs_tags(v_ann.ann_hashcode) }}

Annotated Tags

A {{ iaa_dict.ann[iaa_display_hashcode].anns[0]._filename }}
B {{ iaa_dict.ann[iaa_display_hashcode].anns[1]._filename }}
{{ iaa_gs_dict[iaa_display_hashcode].ann._filename }}
{{ tag_name }} AB: {{ tag_rst.cm.tp }} A+: {{ tag_rst.cm.fp }} B+: {{ tag_rst.cm.fn }}
{{ tag_name }} Total: {{ count_iaa_gs_tag_notnull(iaa_gs_dict[iaa_display_hashcode].rst[tag_name]) }}  
 
Overall Cohen's Kappa: {{ to_fixed(iaa_dict.all.cohen_kappa.kappa) }}     Percentage Agreement: {{ to_fixed(iaa_dict.all.cohen_kappa.Po) }}
TP: {{ iaa_dict.all.cm.tp }}   FP: {{ iaa_dict.all.cm.fp }}   FN: {{ iaa_dict.all.cm.fn }}   N: {{ iaa_dict.all.cohen_kappa.N }}
Po: {{ to_fixed(iaa_dict.all.cohen_kappa.Po) }}   Pe: {{ to_fixed(iaa_dict.all.cohen_kappa.Pe) }}   SEk: {{ to_fixed(iaa_dict.all.cohen_kappa.SE_k) }}   95% CI: {{ to_fixed(iaa_dict.all.cohen_kappa.lower) }}, {{ to_fixed(iaa_dict.all.cohen_kappa.upper) }}
  Annotator B
    {{ tag_def.name }} EMPTY Pb
Annotator A {{ tag_def_row.name }} {{ iaa_dict.tag[tag_def_row.name].cm.tp }}   {{ iaa_dict.tag[tag_def_row.name].cm.fp }} {{ to_fixed(iaa_dict.all.cohen_kappa.Pes.b[tag_def_row.name]) }}
EMPTY {{ iaa_dict.tag[tag_def_col.name].cm.fn }} 0 {{ to_fixed(iaa_dict.all.cohen_kappa.Pes.b['_EMPTY_']) }}
Pa {{ to_fixed(iaa_dict.all.cohen_kappa.Pes.a[tag_def_col.name]) }} {{ to_fixed(iaa_dict.all.cohen_kappa.Pes.a['_EMPTY_']) }}

Error Summary

  • FP: {{ get_razer_rst().err_stat.by_iaa.n_FP }} | FN: {{ get_razer_rst().err_stat.by_iaa.n_FN }} | Error Rate: {{ to_fixed(100 * get_razer_rst().err_stat.by_iaa.error_rate) }}%
  • Accuracy: {{ to_fixed(100 * get_razer_rst().err_stat.by_iaa.accuracy) }}% | Precision: {{ to_fixed(100 * get_razer_rst().err_stat.by_iaa.precision) }}%
  • Recall: {{ to_fixed(100 * get_razer_rst().err_stat.by_iaa.recall) }}% | F1-Score: {{ to_fixed(get_razer_rst().err_stat.by_iaa.f1) }}
  • Median of errors per document: {{ to_fixed(get_razer_rst().err_stat.by_smu.med_n_err_per_doc) }}

Statistics by Error Types

{{ err_cate }}
{{ err_type }}
{{ to_comma(get_razer_n_stat_by_err_type(err_type)) }}

Statistics by Error Label Count

{{ freq - 1 }} Labels Labels
{{ get_razer_rst().err_stat.by_smu.freq_n_err_labels[freq - 1].FP.length + get_razer_rst().err_stat.by_smu.freq_n_err_labels[freq - 1].FN.length }}

Statistics by Concepts

{{ etag_name }}
{{ stat.FP.length + stat.FN.length }}

Top 10 Error Tags

{{ txt }}
{{ get_razer_rst().err_stat.by_txt[txt].FP.length + get_razer_rst().err_stat.by_txt[txt].FN.length }}

File Distribution by Error Count

Error Distribution

Tag Distribution

Color:

Error List

{{ razer_err_list_uids.length }} Tags
{{ seq_num + 1 }}
{{ get_razer_err(uid)._judgement }} {{ get_razer_err(uid).tag }} {{ get_razer_err(uid).text }} {{ get_razer_err(uid).spans }}
{{ e.type }}
{{ err_cate_idx + 1 }} {{ err_cate }}
{{ err_type_idx + 1 }} {{ err_type }}
Use Error Analysis (EA) Web Service

You can use an EA web service to identify the error types automatically. ATTENTION: the tags and their contextual information will be sent to the following URL, please ensure the network security and data privacy.

Use Text Embedding (TE) Web Service

You can use an TE web service to project the text to 2-D panel for visualization. ATTENTION: the tags will be sent to the following URL, please ensure the network security and data privacy.

{{ display_stat_doc_sum_selected.file_name }} - {{ display_stat_doc_sum_selected.tag_name }} {{ stat_doc_sum_selected_tags.length }} tags
{{ tag.id }}
{{ get_tag_spans_text(tag) }}:
{{ tag.text }}
{{ attr.name }}:
{{ tag[attr.name] }} 

Corpus Summary

:
:
{{ stat_item[1] }}
{{ stat_item[1] }}

Annotation Overview

File Name Total {{ tag_def.name }} {{ tag_def.name }}
{{ stat.file_name }} {{ stat.file_name }} {{ stat._total_tags }} {{ stat[tag_def.name] }} {{ stat[tag_def.name] }} {{ stat[tag_def.name] }} {{ stat[tag_def.name] }}

Token Summary

{{ tag_def.name }}
{{ hint_dict[tag_def.name].texts.length }} {{ count_texts_by_stat_fileter(tag_def) }} / {{ hint_dict[tag_def.name].texts.length }}
-
{{ hint_dict[tag_def.name].nc_dict.count }} DOC-LEVEL
{{ fn_v }} {{ fn }}
{{ hint_dict[tag_def.name].text_dict[v_text_info.text].count }} {{ v_text_info.text }}
{{ fn_v }} {{ fn }}

Raw .txt Files

Loading files ...
Please wait for a little while.

Drag and drop the .txt files or the folder in the following box

Loaded {{ tk_medtaggervis_txt_files.length }} text files (.txt).

.txt files / folder

Output .ann Files

Loading files ...
Please wait for a little while.

Drag and drop the .ann files or the folder in the following box

Loaded {{ tk_medtaggervis_ann_files.length }} annotation files.

.ann files / folder

Visualization of Document and Tags

IOB2/BIO Files

Loading files ...
Please wait for a little while.

Drag and drop the IOB2/BIO files or the folder in the following box

Loaded {{ tk_iob2editor_iob_files.length }} files.

IOB2/BIO files / folder

IOB2/BIO File Editor

{{ hovered_tag._annotator }} {{ hovered_tag.tag }}.{{ hovered_tag.id }} {{ hovered_tag.spans }}
{{ tqv_header }}
{{ tqv_footer }}
SCHEMA NAME
ENTITY TAGS - RELATION TAGS - {{ tags.length }} tag(s) | Tag
TAG NAME
ANN.TYPE
ATTRIBUTE
TYPE
DEFAULT
DEFAULT
ITEMS ({{ att.values.length }})

You can click the "New" button to create an empty annotation schema.
Or click the "Open" button to open an existing schema file to edit.

Once the schema design is finished, you can download it as a DTD format file for backup, or use it directly in annotation.
The specification of the annotation schema file can be found in MedTator Wiki.

For more schema samples, you can check MedTator sample schemas on GitHub repo.

Visualizing the annotations ...