The Disqualification Control allows you to control how strict or lenient you can be on the way participants answer the survey.

The quality control system is a built-in feature on the inca Platform, which automatically evaluates and may disqualify participants during surveys. A participant's quality score is determined based on demerit points that they may accumulate, and whether they are disqualified or not depends on the DISQUALIFICATION CONTROL you set for your project.

How to set up the DISQUALIFICATION CONTROL

On the Project Overview page, you can use the disqualification control to set whether participants may be disqualified based on their demerit points:

Strict — participants are disqualified at 5 (or more) demerit points

Normal — participants are disqualified at 10 (or more) demerit points

Disable — no participant will be disqualified

Note: Demerits Points are evaluated differently from how participants answer open-ended questions as well as close-ended questions. For close-ended questions, demerit points are only applicable for Popup Cards and Tradeoff questions.

Open Ended Demerit Points

Users may select any combination of the following checks through the Quality Control tab for each open-ended question:

Questions with any of these checks enabled will have a maximum demerit point influence which you can control the Quality Control tab. You can select either Strong (10 Demerits Points) Normal (5 Demerit Points) or Weak (2 Demerits Points)

If any of the enabled checks are triggered, then the participant will be allocated demerit points. For open-ended questions without probing, this will equal the maximum demerit point influence. When probing is enabled, the maximum demerit point influence is only allocated if all of the responses trigger an enabled check; otherwise, each response contributes a diminishing amount of points, with the prime response having the most influence and each subsequent probe response having less.

Overall these are the main factors that significantly influence the demerit points on open-ended questions:

• Completeness of the Response: If the response is incomplete or consists of gibberish strings (e.g., “xwoiqiojiocjvi”), it is highly likely to result in high demerit points.
  
• Consistency and Appropriateness of Language: If the response is not written in the specified language or differs from the question's language, it is very likely to result in higher demerit points. Additionally, inappropriate words will also generate high demerit points.
  
• Pertinence/Contextual Consistency: If the response is pertinent to the question, it will be treated as high quality, resulting in low demerit points.)
  
• Typographical errors and grammar issues: The demerit scores of flagged participants are also generally affected by typographical errors and grammar issues even though the response is not "bad or low quality"

Close Ended Demerit Points

Close-ended demerit points are included automatically (by default and you can't control the settings) and are currently only available for Popup and Tradeoff question types: straight-lining is detected for each of these, and when a detected participant will be given as many demerit points as there are cards/concepts within the question (e.g. a 7-card popup question will give a participant 7 demerit points).

Quality Control Results

The following results are available in inca dashboard and/or data export:

Quality Score— available in the data export file as well as a dashboard variable. Determined for each participant based on the descriptions below:

• Good Quality — indicates that the participant had a perfect score (no demerit points).
• Disqualified — indicates that the participant was disqualified based on their demerit points.
• Flagged — indicates that the participant did not have a perfect score, but was not disqualified. Demerit Points — only available in data export file, and broken down into the following types:

Demerit Points— only available in data export file, and broken down into the following types:

• Total Demerit Points — number of demerit points accumulated over the whole survey. 
• Close-ended demerit Points — number of demerit points associated with a specific popup or tradeoff question.
• Dialogue Exchange Demerit Points — number of demerit points associated with a participant’s response to each open-ended prime or probe question. 
• Dialogue Demerit Points — number of demerit points accumulated throughout a single dialogue, i.e. associated with one open-ended prime question and all of its subsequent probe questions.