A Quick Guide for Advanced Features

In this guide, we only explain some key concepts and demonstrate advanced features. Throughout our website, most pages have "Read this first", instructional text, and additional tool tips wherever you find this icon . The FAQ page has answers to many questions. Please look at these places for anything not covered here.


Survey, Campaign, and Distribution

We've made great efforts to give survey creators maximum flexibility in handling their surveys and results. At AllCounted, a survey is considered as intellectual work, which is basically a list of questions. The first step in making a survey available to participants is creating a campaign. A campaign decides when a survey is live and who can take it. Once a survey becomes live, you need to choose how to distribute your survey to people, and the options include web links, emails, Facebook links, and embedding in a website. This three-level survey-campaign-distribution structure creates maximum flexibility in targeting survey participants and analyzing responses.

A survey can have multiple campaigns that target the same or different groups of people at the same or different times. Each campaign can have multiple distributions to one or multiple groups of participants. We let you organize survey data at the survey, campaign or distribution level and to analyze survey responses separately, as a whole, or in any possible combination. Additionally, you can compare results between campaigns or distributions through cross tabulation.

For greater flexibility, we allow you to put certain survey responses into one or more labeled groups, which we call Analysis Group. With this tool, you can analyze results by group, or you can even compare results between groups through cross tabulation.

Another benefit of campaigns, distributions, and analysis groups is that they can be used as data groups (samples or datasets) in statistical analysis. When a statistical tool needs more than one data group, not only can you use the answer choices of a Multiple Choice question to split responses and create groups, but also campaigns, distributions, or analysis groups. Moreover, campaigns, distributions, and analysis groups can be treated as a categorical variable used in a Chi-Square test of independence and other statistical tests.

In addition to campaigns, distributions, and analysis groups, you can always use individual questions and information about responses as conditions to slice and dice responses into whatever datasets you'd like for analysis (Tip: You can use Analysis Group or Shortcut to remember a desired dataset). All these conditions are collectively called Filters. The following screenshot shows these filters:

Filters for selecting survey responses for analysis

Skip Rules

Regarding the logic in a survey, we offer question skip rules, question conditions, question scoring rules, page start skip rules, and page end skip rules. Collectively, they are called survey rules. A skip rule on a question redirects a participant to a later page or a question on a later page based on the answer to the question. A question condition allows you to define a situation based on how a question is answered and to combine it with the conditions of other questions on the same page to build a page skip rule. A question scoring rule allows you to assign a score to the answer of a question. You can create multiple skip rules, conditions, or scoring rules for a single question. For scoring rules, our system is able to sum the scores of each question to form the total score of a response in the Analyze section.

A page start skip rule is tested and executed before the skip rules of all questions on the same page. A page end skip rule is tested and executed after all question skip rules on the same page. Page skip rules can combine the conditions of multiple questions, which makes them different from question skip rules. You must create conditions for related questions before using them to create a page skip rule. You can create multiple page start or page end skip rules. The screenshots below show where to find survey rules:

Creating question rules
Creating page skip rules

Note that skip rules are only useful when your survey has multiple pages. In addition, skip rules are only tested and executed when a page is submitted. In another word, a skip rule is unable to skip the questions on the same page.

Here is an example to demonstrate the use of skip rules. In this example, you want to conduct a survey with five pages of questions about how customers feel about your product and service. Suppose that the first page has the following two multiple choice questions and answer choices:

Question 1 How do feel about our product?

  • Great
  • Okay
  • Bad

Question 2 How do feel about our service?

  • Excellent
  • Average
  • Horrible
The two questions on the first page of the survey

Suppose that you want to implement the following skip rules in this sequence:

  • If a participant selects Great for the product question and Excellent for the service question, then the survey skips to Page 3
  • If a participant selects Great for the product question, then the survey skips to Page 4
  • If a participant doesn't select anything for the two questions, then the survey ends

You can achieve the above design by following these steps. First, you must create one skip rule and two conditions on Question 1:

The skip rule on the Product question
Selecting Great as a condition on the Product question
No answer as a condition on the Product question

Second, create two conditions on Question 2:

Selecting Excellent as a condition on the Service question
No answer as a condition on the Service question

Third, create a page start skip rule:

A page start skip rule uses the conditions from two questions

Fourth, create a page end skip rule:

A page end skip rule uses the conditions from two questions

Here's how the page looks when you are done with the above steps:

Following these steps above will achieve your design because page start skip rules are tested before question skip rules, and question skip rules are tested before page end skip rules. We recommend that you add skip rules after you have designed and ordered all the questions and pages properly, and we also recommend that you always test skip rules before asking participants to take your survey.

Scoring Rules

We use an example to show how to use scoring rules. Suppose you're running a store and selling two categories of products, Category A and Category B. You carry different products for each category and each product has different price. For Category A, you have the following three products and prices:

  • Product 1, $30
  • Product 2, $20
  • Product 3, $10

There are also three products for Category B:

  • Product 4, $150
  • Product 5, $50
  • Product 6, $10

Suppose in this survey, you'd like to find out which people are willing to spend more overall in their next visits to your store. You are also interested in those people who intend to buy Product 4. The following screenshot shows the questions in the survey, and we'll allow people to make multiple selections for the Category B question to show the system's capabilities.

The questions in the survey

Now you need to add scoring rules to the questions. For the Category A question, you'd like to assign the actual price as the score for each selection, so you create the following scoring rule when a participant selects Product 1:

The scoring rule for selection of Product 1 on the Category A question

Similarly, you need to add rules for the other two selections. If you want to make sure there's a score for the answers to a question, the scoring rules must cover all possible scenarios. In our case, the Category A question does not require a mandatory answer, so we must add another scoring rule when a participant doesn't select any product for this question. Note that if no rule applies to the answer to a question, then our system does not add any score to the total score of a response. No score for a question or response does not mean zero. It simply means that no applicable scoring rule is found.

The scoring rule applicable when the Category A question is not answered

Here are the scoring rules so far for the Category A question:

All scoring rules on the Category A question

For the Category B question, you'd like to give a score larger than its price if a participant selects Product 4 (the most expensive one) and a smaller score for Product 6 (the cheapest one). Note that scores can be any number you deem appropriate for your survey, including negative numbers and decimals. Suppose you add the following scoring rules for the Category B question:

The scoring rule when Product 4 is selected
The scoring rule when Product 6 is selected

Here are the scoring rules so far on the Category B question:

All scoring rules on the Category B question

For this question, the sequence of scoring rules is important because it allows for multiple selections. If a participant selects both Products 4 and 6, we'll apply the scoring rule for selecting Product 4. Once this rule is applied, the rule for Product 6 will be ignored. For the Category A question, the sequence of scoring rules has no effect because a participant can only select one product, and there is no overlap of covered scenarios between rules.

Once participants respond to your survey, you can see the score along with the answer to each question in a response when you download responses. See the following screenshot for an example. This tool can be very helpful if you use it with other analytical tools that we offer.

Responses with scores

An Example Survey for Analysis

In the rest of the guide, we will focus on the analytical tools and use the following example survey to explain and demonstrate these tools. Suppose that you are working on a survey project with two goals:

  • Understand the relationship between students' sleep and their academic performance
  • See if gender or sports activities play any role

Let's assume that you have the following four questions in your survey:

Question 1 How many hours do you sleep on average, on a nightly basis? Please enter a number only.

Question 2 What is your average GPA, on a 4.0 scale? Please enter a number only.

Question 3 How much time do you spend on sports on a weekly basis?

  • Less than an hour
  • 1 - 2 hours
  • 2 - 4 hours
  • 4 - 6 hours
  • More than 6 hours

Question 4 What is your gender?

  • Female
  • Male

Suppose that you'll accomplish the following tasks:

  • Create a survey with questions that collect quantitative, categorical, and ordinal data
  • Conduct two-way cross tabulation between student gender and time spent playing sports
  • Do a student's t test to see if GPA means are statistically different between female and male students
  • See if gender and time spent playing sports are associated in any way
  • Perform linear and polynomial regression between time spent sleeping and GPA

Questions and Data Types

Each question in a survey collects some data from survey participants. The type of data a question collects determines the analysis available on the question. AllCounted supports three data types:

Data Type Example Analysis
Categorical Hair color: Black, Brown, Blonde, etc. Percentage, cross tabulation, Chi-square test of independence
Ordinal Agreement with an opinion: Strongly agree, Agree, Neutral, Disagree, Strongly disagree Percentage, mean, median, mode, cross tabulation, Chi-square test of independence
Quantitative Age, income, blood pressure, temperature, number of people in a family, balance on a bank account, etc. Mean, median, mode, standard deviation, t tests, ANOVA, linear or polynomial regression

Ordinal data is a special kind of categorical data. We make this distinction only when needed. Otherwise, they are both referred to as categorical data.

For Questions 1 and 2, you want to collect numbers. To enforce this requirement and make analysis on quantitative data possible, you have to indicate that the data type of these two questions is quantitative. Also, we highly recommend that you use validation to require survey participants to only enter numbers (with or without range requirement) because invalid data will be excluded in the analysis. See the following two screenshots:

Select Quantitative as the data type of a Text Box question
Use validation to require participants to enter numbers only

Suppose that you select Ordinal as the data type for Question 3 and Categorical for Question 4. Please see the following two screenshots:

Select Ordinal as the data type of a Multiple Choice question
Select Categorical as the data type of a Multiple Choice question

Analyze Your Survey

Suppose that you have collected enough responses and are ready to perform analysis. Open your survey, click Analyze, Results as shown below:

The analytical tools you can use

The above screenshot shows the types of responses you can use in analysis and the available analytical tools (under View). You can look at and analyze the responses to your survey in different views. The default and perhaps the most important view is Summary. In the Summary view, we list the text answers to a Text Box question. If the question's data is quantitative, you'll see a Statistics link, which gives you some simple statistics on the responses to your question (Note that you can get many more statistics under the Statistics view). Please see the following two screenshot shots:

A listing of answers to Text Box questions in the Summary view
A few simple statistics on quantitative data

In the Summary view, for Multiple Choice questions like Question 4, we display the statistics in a chart and table as shown below. The chart and the table have the same information, but the table is helpful if the text is too long to display on the chart.

The Summary view of categorical data

Question 3 is special because its data is ordinal. In the Summary view, you can see more statistics such as the mean, median, and mode. Please see the following screenshot. Note that we calculate the mean for ordinal data for practical reasons. We automatically assign 1, 2, etc. to the first row (Less than an hour), second row (1 - 2 hours), etc. when calculating these statistics.

The Summary view of ordinal data

Understand Your Data First

Before you start doing advanced analysis, it's always good to have a basic understanding of the responses to the question you want to analyze. Another reason to review the responses first is because some statistical tests assume that data has met certain requirements; otherwise, an analysis could be meaningless. For example, student's t tests assume that data is normally distributed. A basic understanding of your data can help prevent this type of error.

For quantitative Text Box questions, we offer a wide range of basic statistics such as quartiles, central tendency, dispersion, and distribution. You can even split the data into a number of groups of equal percentile or find a specific percentile. In addition, you can create histograms or boxplots for quantitative responses. To do this basic analysis, click Analyze, Results, and then Exploring Quantitative Data under the Statistics view. You'll be asked to select the variable to analyze. The following two screenshots show this type of analysis on Question 2 (the GPA question).

Basic statistics for responses to a quantitative Text Box question

Histogram and boxplot of responses to a quantitative Text Box question

Cross Tabulation

You have four ways to do cross tabulation: by the answer choices of a Multiple Choice question, campaigns, distributions, or analysis groups. Please see the following screenshot:

Cross tabulation options

For example, let's select "By a Multiple Choice Question" and click the Apply button. You'll be prompted to select a Multiple Choice question. Click the Question menu and select a Multiple Choice question. Suppose we selected Question 4 and its two answer choices, as shown below:

Select a Multiple Choice question and its answer choices in cross tabulation

Click the Apply button, and you'll see the data separated by the answer choices of the selected Multiple Choice question.

Cross tabulation between two categorical questions

The data in the above screenshot is equivalent to the following two tables (The second table is only applicable to Multiple Choice questions with the ordinal data type.):

Less than an hour 1 - 2 hours 2 - 4 hours 4 - 6 hours More than 6 hours
Female 1, 11.11%, 5.88%, 5.88% 2, 22.22%, 11.76%, 11.76% 3, 33.33%, 17.65%, 17.65% 2, 22.22%, 11.76%, 11.76% 1, 11.11%, 5.88%, 5.88%
Male 1, 12.50%, 5.88%, 5.88% 3, 37.50%, 17.65%, 17.65% 4, 50.00%, 23.53%, 23.53%
Mean, Median, Mode
Female 3.00, 3, 2 - 4 hours
Male 3.25, 3.5, 4 - 6 hours

If the data type of a Text Box question is quantitative, then you'll see how its data differs between answer choices. The following screenshot shows how the data of Question 1 varies depending on the answer choices of Question 4.

Comparison of means of sleep hours by gender

The data in the above screenshot is equivalent to the following table:

Mean of Sleep Hours
Female 5.50
Male 4.81

Independent Samples Student's T Test

If you want to compare responses of different groups based on the same set of questions, then you need to create a single survey and collect responses through different campaigns or distributions. Each campaign or distribution counts as one group of responses. You may also separate responses into different groups based on the answer choices of a multiple choice question. In this situation, each answer choice is a label for a group of responses, and you don't need to create multiple campaigns or distributions. Note that AllCounted is unable to compare responses from two surveys even if their questions are exactly the same.

Student's t test is a commonly used tool for comparing the means from different groups of responses. AllCounted supports one sample, independent samples, and paired samples student's t tests. In this guide, we use the independent samples t test as an example. Suppose that you want to know whether the difference in GPA means between female and male students is statistically significant. Click Independent Samples Student's T Test under the Statistics view, and then click the Apply button. You'll be asked to select the variable and data groups. Here are our selections for this demo:

Data groups and settings of independent samples student's t test

The following screenshot shows the results, which indicate that the difference in GPA means is not statistically significant.

Results of independent samples student's t test

Here is a neat feature for survey creators. If you'd like to export the data points in this test and put them in another analytical tool, you can click on the name of a data group to show all the data points and copy all of them with a mouse click. This feature is available for all statistical tools. The following screenshot is an example:

Obtaining the data points used in a test

It is worth noting that for the two data groups in this test, you can also select two campaigns, two distributions, or two analysis groups. This applies to all the statistical tools that need two or more data groups. Besides, you can always use the Filters to further segment responses into the dataset you'd like. The possibilities are endless!

Chi-Square Test of Independence.

To conduct a Chi-square test of independence, you need to have two categorical variables. In our example, the gender and time spent playing sports questions meet this requirement. Click Chi-Square Test of Independence under the Statistics view, then click the Apply button. You'll be asked to select the variables. Here are our selections for this demo:

Settings of a Chi-square test of Independence

The following screenshot shows the results based on the above settings.

Results of a Chi-square test of independence

Note that in this test, you are not asked to select any data group, which means that you have only one group, which is defined by your "Filter by" settings. for this and other tests such as the one-way ANOVA we offer, you can always use filters to slice and dice data, and the results will be updated instantly.

One-Way ANOVA

Doing one-way ANOVA is similar to doing an independent samples t test. The only difference is that you can select only two data groups for the t test, but you can select more than two groups in the one-way ANOVA test. To do one-way ANOVA, click One-Way ANOVA under the Statistics view, then click the Apply button. You'll be prompted to select two or more data groups and the dependent variable. The following two screenshots show our selections in this demo and the results:

Settings of one-way ANOVA

Results of one-way ANOVA

Note that our system is very flexible and you can select any two or more answer choices to form the data groups needed for this test.

Regression

Regression can be performed between two Text Box questions with the quantitative data type. To perform regression between Questions 1 and 2, select "Simple Linear Regression" or "Polynomial Regression" under the Statistics view and click the Apply button. You'll be prompted to select the independent and dependent variables. Let's select Question 1 as the independent variable. Click the Apply button, and you'll get the following table, function, and chart:

Linear regression between two quantitative Text Box questions

For polynomial regression, you can optionally specify the degree of the fitting polynomial function. You may enter a number between 1 and 20 for the degree. The higher the degree, the more time it'll take to produce the regression function and chart. If you don't select the degree, we'll use 2 by default. The following screenshot shows this:

Select a degree for polynomial regression

Results of polynomial regression

Data Export

If our analytical tools can't satisfy your needs, you can export all or some of your survey responses into a text, xml, or Excel file. Click the Results menu, select Individual Responses, and click the Apply button, which will display a list of individual responses. Then, click the Export menu and the download format. The download will start immediately. Please note that if you have a lot of responses, the download may take a while. The good news is that you can leave the download window open and start another browser window to continue using our website without waiting for the download to finish.

Download individual responses to a survey into a file

As mentioned above, in statistical tools, you can click the name of a data group and export all the data points used in a test.

We can help

We offer many powerful and innovative tools for online surveys, and you need to sign up to use them. All tools and services at AllCounted are completely free. If you have any questions, suggestions or see something that's unclear, missing, or incorrect, please don't hesitate to contact us at support@allcounted.com. We will do our best to timely serve your needs.


If your colleagues, friends, or classmates need a tool for online surveys, please let them know that we can help. Thank you!