What is a Tradeoff Ranking Question and how to set it up?

What is a Tradeoff Ranking Question and how to set it up?

Main use:

Tradeoff Ranking is mainly used to test the attractiveness and competitiveness of different ideas, including concepts, messages, logos, features and other stimulus materials. It explores further why the customers like or dislike the ideas so that users can understand from their customers’ perspective and iterate their ideas accordingly.

Tradeoff Ranking consists of three parts.

The first part asks the participants to rate if they like or dislike the idea presented in a series of cards (Carousel)

  • Ideas will be shown one by one in a randomized order
  • Here users can define the scales based on their needs - For example, they can choose a scale with three points for measuring liking, such as like, not sure, and dislike; Or they can use a scale with five points for gauging interest level, ranging from not at all interested to very interested.

The second part asks the participant to make head-to-head comparisons among the ideas that they liked in the first part until we identify the one that the person likes the most (i.e. the Top Choice). 

  • Head-to-head comparisons are made only among the ideas liked in the first part. If it’s a 5 scale point, then all the ideas with a rating of 4 and 5 would go to the head-to-head comparison.
  • If there’s only one idea they liked in the first part, then this one will automatically become the Top Choice without head-to-head comparison.
  • If they don’t like any of the ideas, then this part will be skipped and they will be asked why they dislike one of the ideas.

The third part includes two follow-up open-ended questions as to why they like or dislike the idea.

  • “Why they like” will be asked for the Top choice
  • “Why they dislike” will be asked for one of the ideas they dislike (We would randomly select one idea if there are multiple ones that they don’t like)

A quick video to show how Tradeoff Ranking works on the inca chatbot


How to set up in the platform:


1. In the Questionnaire Builder page, add new question OR insert new question by hovering over an existing question and click the green (+) button (either above or below the existing question).



2. Select Tradeoff Ranking from the popup menu that appears. 


​

​
3. Write your question number or label in the upper box 




​

4. Confirm or edit the Rating Question Text

(Note: The Rating Question's objective is to ask the participants to rate if they like or dislike the idea/concept presented in a Carousel or series of cards)

5. Confirm or edit the Head-to-Head Comparison Question Text

(Note: The Head-to-Head Comparison Question's objective is ask the participants to make head-to-head comparison (which one they like better) among the ideas/concepts that they liked in the first part)



​

6. Add the concept stimuli in the Cards box




Cards (Content):

- If adding the attributes or statements by typing one by one in the Option Content field, click Add Card

- If adding attributes or statements using copy-paste from a list, or drag-and-drop list/image, click Batch Add

- When adding Cards (whether individually or batch add) the Card Label (what appears in result visualization or dashboard) will also be automatically updated, but can also be modified by users. The Value or card codes cannot be modified.




Cards (Settings):

A. Card Format
- The users can specify the card formats and choose from 'Text only' or 'Image only' 

B. AutoFill Card Values
- If enabled, the card values will be auto filled based on the options (1 to N)




​
7. In the Response Options Box, select the Option types:
  • 2-pt scale: Likes, Dislike
  • 3-pt scale: Likes, Not sure, Dislike
  • 5-pt scale: Dislike very much, Dislike, Not sure, Like, Like very much

    Note: Users can modify the Option Content and Image





​

8. Follow-up questions are enabled for the two open-ended questions Why Positive Question Text and Why Negative Question Text. Probing is also enabled for these two follow-up questions.
  1. Why Positive Question Text : This question is asked only for the most preferred concept/idea.
  2. Why Negative Question Text: If more than 1 concept is disliked, this follow-up question will be only asked randomly for one of the disliked concept.




As default, there will be no follow-up questions for concepts/ideas with Not sure answers or Neutral response. 
(Note: You can use the sample questions or edit the wording for the two questions. The objective is to ask why they like the idea/concept (above the rest) for Why Positive Question; and ask why they dislike or less prefer the idea/concept from others for Why Negative Question)



Other Useful Question Settings




Allow User Corrections

To allow participants to update or change answer after submitting, toggle on Allow User Corrections. 


    • Related Articles

    • What is a Virtual Question and how to set it up?

      Main use: Virtual Question is a question that is hidden from the real survey conversation. With this question type, we can create variables that combine different options from other questions. Those variables can eventually help with the survey logic ...

    • How to add QUESTION CONDITIONS?

      In editing your survey, you may want to ask questions with logic based on previous answers or redirect to a question based on how they answer a certain question. Adding QUESTION LOGIC, allows the user to control the questionnaire flow/routing in the ...

    • What question types are available on inca?

      inca provides 20 different types of questions, which fold into 4 main categories -Standard Questions, Specialty Questions, Static Content and Logic Questions. Presented with an image of a Blob Tree, participants are asked to select a character that ...

    • What is ANSWER REFERENCING and how to set it up?

      With ANSWER REFERENCING, the user can customize the wording for each participant by referencing the answers from a previous question. Answer referencing can be added to the toolbox for question wording. You may choose to reference TEXT format answer ...

    • How to set up a Rank Order Question?

      Main use: Using this question type, participants can rank items or statements in order of importance or preference (e.g. most important to least important). How to set up the platform: 1. In the Questionnaire Builder page, add a new question OR ...