About the AMR Checker
Written by Ulf Hermjakob, May 25, 2013 |
Purpose
The purpose of the AMR Checker is to support the annotator in building high-quality AMRs and to help assure consistency among different annotators by identifying possible errors and bringing annotation alternatives to the attention of the annotator.
Some typical examples are:
- An English word should have been annotated using a verb, e.g. a teacher should have been annotated as a person who teaches.
- The argument structure of a verb does not fit its frame, e.g. the AMR was annotated with an :ARG0 for a specific verb, but the verb frame for that verb only has :ARG1 and :ARG2.
- A role is generally invalid, or there is a better role for a specific case.
- The structure of an AMR does not conform with the guidelines, e.g. for a named entity.
- Something is missing in the AMR.
- There's a typo in the AMR.
Example
isi_0002.210 Roosevelt announced his decision to run for reelection .
(a / announce-01
:ARG0 (p / person
:name "Rosevelt")
:ARG1 (d / decide-01
:ARG1 (r / run-02 N
:ARG1 p
:ARG1 (r3 / reelection))))
| What does this mean?
- Typo: Rosevelt. Should be: Roosevelt.
- Forgot to annotate his. Add :ARG0 p under decide-01.
- Name structure collapsed. Should be (p / person :name (n / name :op1 "Roosevelt")) instead of just (p / person :name "Roosevelt")
- run-02 has two :ARG1s. First one should be :ARG0.
- reelection should be verbalized as reelect-01 (with argument :ARG1 p).
|
Note: Mouse over English words or AMR elements to see automatically generated alignments.
False alarms
Many of the AMR Checker error messages and warnings are due to actual errors in the AMR,
but some of the flagged AMRs are ok, particularly in the sentence coverage component
where the checker tries to align English words to concepts, strings or roles of the AMR,
but might not always have enough linguistic resources available to properly make such an alignment.
Typical examples of false alarms:
- Some function words such as and, into, or should are flagged, but for a specific sentence, they don't need to be represented directly in the AMR.
- The AMR uses a non-literal way to capture an English expression, e.g. imprison-01 for "to be behind bars".
- In sentences with quotations such as "I am hungry," said the little prince. the AMR Checker
might signal that the first word I could not be matched, but the AMR,
rather than using a literal (i / i) has correctly co-indexed the I with the p
of the little prince.
- Typo in the English sentence, but correct AMR.
Here is an example of false alarms, the AMR Checker results applied to thoroughly edited consensus guideline AMRs.
Remember that the AMR Checker only makes suggestions, the annotator makes the final decision:
- I saw a dead cockroach (= a cockroach that died) In this case, the suggestion to map dead → die-01 is reasonable.
- He was dead wrong (= completely wrong) In this case, dead has a different meaning, and the suggestion to map dead → die-01 should not be followed.
Easy way to make corrections
If the annotator has write permission for a workset (normally his/her own worksets), the AMR Checker
will place an "Edit this AMR" button next to each AMR for convenient editing access to that AMR.
After editing an AMR the AMR Checker page is not
automatically regenerated, but the "Edit this AMR" button of the visited AMR will change to a
darker blue to indicate that that AMR has previously been revisited.
AMR Checker location
| The AMR Checker is available in the AMR Editor, where you will find a | | button on the bottom left. |