The goal of the QA lab (kickoff event slide) is to provide a module-based platform for system performance evaluations and comparisons for solving real-world university entrance exam questions. In its first year at NTCIR 11, world history questions are selected from The National Center Test for University Admissions and from secondary exams at 4 universities in Japan. Both Japanese and English translations of the topics (questions) are provided in an XML format. Participants are welcome to use any resources except solutions to the exams.
Some of the highlights are:
1. Solving real-world problems.
2. Many questions are not in a simple QA format, and require an understanding of the surrounding context.
3. Some questions require inference.
At NTCIR 11, most of the questions are True/False questions or factoid questions, with some questions involving short answers of 50 to 300 Japanese characters. In the next round at NTCIR 12, the plan is to include biology questions and increase the difficulty and the number of complex questions.