NTCIR

NII Testbeds and Community for Information access Research
  • NTCIR Official site
  • Conference
  • Organizers
  • Data
  • Important Dates
 

NTCIR-11 QA-Lab

updated on 2014-09-08

Gold Standard Format


Gold Standard for training data is available from the Download website, whose URL was delivered to each participating team after submitting the signed user agreement forms. Gold Standard of testing (Formal run) data will be delivered after each phase of formal runs.

DTD

"answerTable.dtd" is available from the Download webliste for each participating Team although it is refered its URL in the XML file.


Sample XML Format

A full XML files are available in the training data, "answer-0625training-center_test(1997,2001,2005,2009).zip" file, which is available at the Download webliste for each participating Team. The below is an abridged sample for the GoldStandard (right answer).  The submission format for the final results is the same.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE answerTable SYSTEM "http://21robot.org/answerTable.dtd">
<answerTable filename="Center-2009--Main-SekaishiB">
<data>
<section>第1問</section>
<question>1</question>
<answer_column>1</answer_column>
<answer>3</answer>
<score>2</score>
<answer_type>sentence</answer_type>
<answer_style>multipleChoice</answer_style>
<knowledge_type>KS</knowledge_type>
<question_ID>Q2</question_ID>
<anscolumn_ID>A1</anscolumn_ID>
</data>
<data>
<section>第1問</section>
<question>2</question>
<answer_column>2</answer_column>
<answer>1</answer>
<score>3</score>
<answer_type>sentence</answer_type>
<answer_style>multipleChoice</answer_style>
<knowledge_type>KS</knowledge_type>
<question_ID>Q3</question_ID>
<anscolumn_ID>A2</anscolumn_ID>
</data>
<data>
<section>第1問</section>
<question>3</question>
<answer_column>3</answer_column>
<answer>1</answer>
<score>3</score>
<answer_type>(symbol-term_other)*2</answer_type>
<answer_style>multipleChoice</answer_style>
<knowledge_type>KS</knowledge_type>
<question_ID>Q4</question_ID>
<anscolumn_ID>A3</anscolumn_ID>
</data>
<data>
<section>第1問</section>
<question>4</question>
<answer_column>4</answer_column>
<answer>4</answer>
<score>3</score>
<answer_type>sentence</answer_type>
<answer_style>multipleChoice</answer_style>
<knowledge_type>KS</knowledge_type>
<question_ID>Q5</question_ID>
<anscolumn_ID>A4</anscolumn_ID>
</data>
<data>
<section>第1問</section>
<question>5</question>
<answer_column>5</answer_column>
<answer>1</answer>
<score>3</score>
<answer_type>sentence</answer_type>
<answer_style>multipleChoice</answer_style>
<knowledge_type>KS</knowledge_type>
<question_ID>Q6</question_ID>
<anscolumn_ID>A5</anscolumn_ID>
</data>
<data>
<section>第1問</section>
<question>6</question>
<answer_column>6</answer_column>
<answer>3</answer>
<score>3</score>
<answer_type>sentence</answer_type>
<answer_style>multipleChoice</answer_style>
<knowledge_type>KS</knowledge_type>
<question_ID>Q7</question_ID>
<anscolumn_ID>A6</anscolumn_ID>
</data>
<data>
<section>第1問</section>
<question>7</question>
<answer_column>7</answer_column>
<answer>2</answer>
<score>3</score>
<answer_type>sentence</answer_type>
<answer_style>multipleChoice</answer_style>
<knowledge_type>KS</knowledge_type>
<question_ID>Q8</question_ID>
<anscolumn_ID>A7</anscolumn_ID>
</data>
<data>
<section>第1問</section>
<question>8</question>
<answer_column>8</answer_column>
<answer>2</answer>
<score>3</score>
<answer_type>sentence</answer_type>
<answer_style>multipleChoice</answer_style>
<knowledge_type>KS</knowledge_type>
<question_ID>Q9</question_ID>
<anscolumn_ID>A8</anscolumn_ID>
</data>
<data>
<section>第1問</section>
<question>9</question>
<answer_column>9</answer_column>
<answer>4</answer>
<score>2</score>
<answer_type>sentence</answer_type>
<answer_style>multipleChoice</answer_style>
<knowledge_type>KS</knowledge_type>
<question_ID>Q10</question_ID>
<anscolumn_ID>A9</anscolumn_ID>
</data>
<data>
<section>第2問</section>
<question>1</question>
<answer_column>10</answer_column>
<answer>4</answer>
<score>3</score>
<answer_type>(symbol-term_location)(symbol-term_other)</answer_type>
<answer_style>multipleChoice</answer_style>
<knowledge_type>KS</knowledge_type>
<question_ID>Q12</question_ID>
<anscolumn_ID>A10</anscolumn_ID>
</data>
<data>
<section>第2問</section>
<question>2</question>
<answer_column>11</answer_column>
<answer>3</answer>
<score>3</score>
<answer_type>sentence</answer_type>
<answer_style>multipleChoice</answer_style>
<knowledge_type>KS</knowledge_type>
<question_ID>Q13</question_ID>
<anscolumn_ID>A11</anscolumn_ID>
</data>
<data>
<section>第2問</section>
<question>3</question>
<answer_column>12</answer_column>
<answer>4</answer>
<score>3</score>
<answer_type>sentence</answer_type>
<answer_style>multipleChoice</answer_style>
<knowledge_type>KS</knowledge_type>
<question_ID>Q14</question_ID>
<anscolumn_ID>A12</anscolumn_ID>
</data>
<data>
<section>第2問</section>
<question>4</question>
<answer_column>13</answer_column>
<answer>4</answer>
<score>3</score>
<answer_type>sentence</answer_type>
<answer_style>multipleChoice</answer_style>
<knowledge_type>KS</knowledge_type>
<question_ID>Q15</question_ID>
<anscolumn_ID>A13</anscolumn_ID>
</data>
</answerTable>