What is Project-Level Conversational Settings and how to set it up?
In the Question Builder, users can control and apply the same Quality Control and Smartprobe settings for all Open End questions in their study. This can be done in the Overview page using the Conversational Settings tab.
Once users enable the Project-Level Conversational Settings, the following main settings for all Open End questions can be controlled at project level:
1. Quality Control Setting
The quality control system is a built-in feature on the inca Platform, which automatically evaluates and may disqualify participants during surveys.
To learn more about best practices for each Quality Control Setting i.e. Gibberish Detection, Uninformative Answer Detection, Duplicate Detection and AI-Generated Content Detection, please refer to this article here
2. Smartprobe Settings
The SmartProbe settings can intelligently generate follow-up questions ("probes"), assess whether their answers warrant demerit points, and can also detect conversational targets of interest.
To learn more about best practices and use case for each Smartprobe Setting for Research Context, Conversational Targets, Multi-Turn probing and please refer to this article here
1. Quality Control Setting
The quality control system is a built-in feature on the inca Platform, which automatically evaluates and may disqualify participants during surveys.
To learn more about best practices for each Quality Control Setting i.e. Gibberish Detection, Uninformative Answer Detection, Duplicate Detection and AI-Generated Content Detection, please refer to this article here
2. Smartprobe Settings
The SmartProbe settings can intelligently generate follow-up questions ("probes"), assess whether their answers warrant demerit points, and can also detect conversational targets of interest.
To learn more about best practices and use case for each Smartprobe Setting for Research Context, Conversational Targets, Multi-Turn probing and please refer to this article here
To override any Quality Control and/or Smartprobe settings for any individual Open End question, user will have to control this on the Conversational Settings tab for each applicable Open End question.
Note:
Maximum Demerit Points/Influence
inca’s demerit system flags respondents based on several factors for open-ended responses, including completeness of the response, consistency and appropriateness of language, pertinence/contextual consistency and typographical errors and grammar issues.
Questions with any of these checks enabled will have a maximum demerit point influence which you can also control on a project level. You can select either Strong (10 Demerits Points), Normal (5 Demerit Points) Weak (2 Demerits Points) or Disable (No influence).
Participants will accumulate demerit points as they go through the entire survey. Users have the ability to control for all open-ended question, what kind of influence they want the demerit to have on terminating people during the fieldwork, or they can also choose to only flag the respondents but not terminate during the fieldwork and use the Quality Score as an indication for QC after data collection. For close-ended questions, demerit points are only applicable for Popup cards and trade-off questions to identify straightliners.
Let's say you have 5 Open End questions in your study...
1. When you set the demerit influence to ‘Weak – 2 points’ for all OEs, and Disqualification Control to ‘Normal – 10 points’ in the General tab of Overview page, a participant will have to give 5 very bad open end responses to be terminated (assuming they are not straightliners).
2. When you set the demerit influence to "Disable" for all OEs, and the Disqualification to 'Normal - 10 points', any bad response from the participant will not be terminated as all the OEs have no influence on quality score. Nevertheless, you can use the Quality Score as indication for QC after data collection
3. When you set the demerit influence to ‘Strong – 10 points’ for all OEs, and Disqualification Control to ‘Normal – 10 points’ in the General tab of Overview page, a participant will only have to give 1 very bad open end response and they will be immediately terminated (assuming they are not straightliners).
2. When you set the demerit influence to "Disable" for all OEs, and the Disqualification to 'Normal - 10 points', any bad response from the participant will not be terminated as all the OEs have no influence on quality score. Nevertheless, you can use the Quality Score as indication for QC after data collection
3. When you set the demerit influence to ‘Strong – 10 points’ for all OEs, and Disqualification Control to ‘Normal – 10 points’ in the General tab of Overview page, a participant will only have to give 1 very bad open end response and they will be immediately terminated (assuming they are not straightliners).
Related Articles
How to set up a Rating Question?
Main use: Similar to a choice question but by default the participants can only select for a visual scale of icons: stars, hearts, thumbs up. The number of icons represents the level of agreement, importance, preference or likelihood. How to set up ...How to set up an Open Ended Question and AI Probing?
Main use: Using an open-ended question, participants can freely type answers in their own words (as opposed to a pre-determined list in Choice questions), which can be in lengthy essays or short texts. How to set up the platform: 1. In the ...How to set up Audience from SELF-PROVIDED sources?
What is a SELF-PROVIDED audience? When the user prefers to use their audience sources instead of directly using the inca panel, it is referred to SELF-PROVIDED audience. SELF-PROVIDED audience mainly includes - Any third-party online panel(s) that ...How to set up Audience from INCA PANEL
What is INCA PANEL? INCA PANEL connects to Dynata's global panel, through which, the user can source Audience directly from inca platform without reaching out to the panel providers separately. How to set it up? In order to set up the Audience from ...How to set up projective technique using Guided Fantasy?
Main use: Guided Fantasy is an animated projective technique that allows participants to elicit how they feel about a brand (Planet X brand). With Guided Fantasy, participants are asked to imagine they are on a spaceship travelling to Planet X (e.g. ...