NTCIR

NII Testbeds and Community for Information access Research
  • NTCIR Official site
  • Conference
  • Organizers
  • Data
  • Important Dates
 

SEPIA

updated on 2014-09-29


Introduction

SEPIA is a web based application where:

  • Topic Developers can develop topics (questions) and their gold standard for IR/QA task

  • Task Participants can download files (topics file, xmi and xml run result filess, evaluation results etc) and upload files (xmi and xml run result files, system description file etc)

  • Assessors can evaluate submitted IR/QA run results

This page describes capabilities of SEPIA for Task Participants.

How to Download

Before accessing SEPIA, make sure Javascript is enabled in your web browser.

  1. Go to http://136.187.114.90/sepia/index.jsp

  2. Enter your team username and password. In case you haven't received one, email qalab-admin [at] nii.ac.jp.
  3. Select the task from the pull down list, and click on "select" button. For Phase 1, there are two tasks ; (1) Phase 1 English Center Exam, and (2) Phase 1 Japanese Center Exam.
  4. Go to the download page
  5. You will see a list of files you can download, including files you uploaded before.
  6. Select files to download, then click on "Download Selected Files" button

How to Upload

Please send the run results by email to the task organizers at qalab-admin@nii.ac.jp. 
The following uploader doesn't work.

Before accessing SEPIA, make sure Javascript is enabled in your web browser.

First of all, follow the steps 1-3 above.

  1. Go to http://136.187.114.90/sepia/index.jsp

  2. Enter your team username and password. In case you haven't received one, email qalab-admin [at] nii.ac.jp.
  3. Select the task from the pull down list, and click on "select" button. For Phase 1, there are two tasks ; (1) Phase 1 English Center Exam, and (2) Phase 1 Japanese Center Exam. 
  4. Go to the upload page
  5. Select the file type from one of "Question Analysis (XMI and XML)", "IR (XMI and XML)", "Information eXtraction (XMI)", "End-to-End QA (XML)", "Combination (XML)", and "Other". Select "Other" when you submit a system description file. For XML files, type specific validation runs.
  6. Specify the file and click on the submit button.
  7. You will see a success message if validation passes. In case validation fails, fix the problem according to the error message.

FAQ

Q1. What should I do when the SEPIA doesn't respond?
A2. First, try "super-reload" by Ctrl + F5. If there's still a problem, contact qalab-admin [at] nii.ac.jp

Q2. Where can I download the Phase 1 Formal Run Test data?
A2. The Phase 1 Formal Run Test data is available at your team's data access page on the "warehouse" server, where you had downloaded your training data. Please reload the page and download the data.

Q3. What should I do when you have any problem to upload the run results to SEPIA.
A3. Please send them by email to the task organizers at qalab-admin@nii.ac.jp.