2. Task Description
A single task is carried out in two separate cycles. Each cycle has a separate training set and test set with similar difficulty. The task is to return solutions that could be either True/False or short answers given a list of topics (questions) in an XML format.
The module-based platform means participants can participate in any phase of QA system development. Our module is divided into 4 phases, question analysis, document retrieval, extraction of answer candidates, and answer generation. Participants are also welcome to skip the entire process and simply submit their solutions. At the end of each phase, submissions are evaluated, and participants are free to use their own submission in the current phase or baseline provided by the organizer to continue on to the next phase, or stop altogether. This means it is possible to just participate in the document retrieval phase of a single cycle of QA lab.