Each user is presented to a random series of questions. Per each answered question, the labeler is rewarded with a different amount of points depending on how accurate it has been up to the last question. The accuracy score of each labeler is updated only with certain objects (for which the research group know the ground truth).

As labeler you will have some examples and help about how to respond accurately. So, do not worry about your previous knowledge and try your best guess!

To learn more about how to be prepared go to: Learn more.