NTCIR

NII Testbeds and Community for Information access Research
  • NTCIR Official site
  • Conference
  • Organizers
  • Data
  • Important Dates
 

NTCIR-11 QA-Lab

updated on 2014-09-30


4. Submission

QA-Lab participants will submit the followings for Phase 1:

 File Who should submit
 Submission
restriction
 Format
(recommended)
 Format
(acceptable)
Phase 1 Due
(23:59 JST)
 Question Analysis results
 QA participants
 up to 3 runs will be evaluated
 XMI
XML 2014-09-30
 IR run results (RS)
 IR participants
  up to 3 runs will be evaluatedXMI XML 2014-09-30
 Information Extraction results (IX)
 QA participants
   up to 3 runs will be evaluated XMI -- 2014-09-30
 End-to End QA run results
 QA participants
  up to 3 runs will be evaluated XML 2014-09-30
 Combination (IR+QA) run results
 QA participants
(recommended)
 up to 3 runs will be evaluated
 XML 2014-10-06
 UIMA Modules or API for non-UIMA modules
 participants TBATBA 2014-09-30
 System Description
 ALL participants
(required)
 TBA TBA 2014-10-06

For the module run results, XMI is recommended and can be used in the UIMA-based systems. The XML files are acceptable only those who can not create XMI output. The XMI files can be (roughly) converted into XML, but XML can not be converted into XMI.
The formals for the XML interrim results from the modules (question analysis, retrieval) are followed those for the NTCIR-8 ACLIA's "Question Analysis" and "IR4QA" formats.

Regarding more details about the format and restriction, please consult the following pages carefully before running the formal run topics:

  • Format - Submission format specification, restriction, etc.
  • SEPIA - How to upload and download the QA-Lab related files using SEPIA