TDSM 3.13
From The Data Science Design Manual Wikia
We can put some obvious examples that we know the answers randomly in the data as evaluation texts. Then check the answer of each person for those texts. If someone has annotated randomly, it is probable that s/he answered the evaluation texts randomly too.