A Quick Guide for Advanced Features
In this guide, we only explain some key concepts and demonstrate advanced features. Throughout our website, most pages have "Read this first", instructional text, and additional tool tips wherever you find this icon . The FAQ page has answers to many questions. Please look at these places for anything not covered here.
Survey, Campaign, and Distribution
We've made great efforts to give survey creators maximum flexibility in handling their surveys and results. At AllCounted, a survey is considered as intellectual work, which is basically a list of questions. The first step in making a survey available to participants is creating a campaign. A campaign decides when a survey is live and who can take it. Once a survey becomes live, you need to choose how to distribute your survey to people, and the options include web links, emails, Facebook links, and embedding in a website. This three-level survey-campaign-distribution structure creates maximum flexibility in targeting survey participants and analyzing responses.
A survey can have multiple campaigns that target the same or different groups of people at the same or different times. Each campaign can have multiple distributions to one or multiple groups of participants. We let you organize survey data at the survey, campaign or distribution level and to analyze survey responses separately, as a whole, or in any possible combination. Additionally, you can compare results between campaigns or distributions through cross tabulation.
For greater flexibility, we allow you to put certain survey responses into one or more labeled groups, which we call Analysis Group. With this tool, you can analyze results by group, or you can even compare results between groups through cross tabulation.
Another benefit of campaigns, distributions, and analysis groups is that they can be used as data groups (samples or datasets) in statistical analysis. When a statistical tool needs more than one data group, not only can you use the answer choices of a Multiple Choice question to split responses and create groups, but also campaigns, distributions, or analysis groups. Moreover, campaigns, distributions, and analysis groups can be treated as a categorical variable used in a Chi-Square test of independence and other statistical tests.
In addition to campaigns, distributions, and analysis groups, you can always use individual questions and information about responses as conditions to slice and dice responses into whatever datasets you'd like for analysis (Tip: You can use Analysis Group or Shortcut to remember a desired dataset). All these conditions are collectively called Filters. The following screenshot shows these filters:
Regarding the logic in a survey, we offer question skip rules, question conditions, question scoring rules, page start skip rules, and page end skip rules. Collectively, they are called survey rules. A skip rule on a question redirects a participant to a later page or a question on a later page based on the answer to the question. A question condition allows you to define a situation based on how a question is answered and to combine it with the conditions of other questions on the same page to build a page skip rule. A question scoring rule allows you to assign a score to the answer of a question. You can create multiple skip rules, conditions, or scoring rules for a single question. For scoring rules, our system is able to sum the scores of each question to form the total score of a response in the Analyze section.
A page start skip rule is tested and executed before the skip rules of all questions on the same page. A page end skip rule is tested and executed after all question skip rules on the same page. Page skip rules can combine the conditions of multiple questions, which makes them different from question skip rules. You must create conditions for related questions before using them to create a page skip rule. You can create multiple page start or page end skip rules. The screenshots below show where to find survey rules:
Note that skip rules are only useful when your survey has multiple pages and that a skip rule on a page is unable to skip the questions on the same page.
Here is an example to demonstrate the use of skip rules. In this example, you want to conduct a survey with five pages of questions about how customers feel about your product and service. Suppose that the first page has the following two multiple choice questions and answer choices:
Question 1 How do feel about our product?
Question 2 How do feel about our service?
You want to implement the following skip rules in this sequence:
- If a participant selects Great for the product question and Excellent for the service question, then the survey skips to Page 3
- If a participant selects Great for the product question, then the survey skips to Page 4
- If a participant doesn't select anything for the two questions, then the survey ends
You can achieve the above design by following these steps. First, you must create one skip rule and two conditions on Question 1:
Second, create two conditions on Question 2:
Third, create a page start skip rule:
Fourth, create a page end skip rule:
Here's how the page looks when you are done with the above steps:
Following these steps above will achieve your design because page start skip rules are tested before question skip rules, and question skip rules are tested before page end skip rules. We recommend that you add skip rules after you have designed and ordered all the questions and pages properly, and we also recommend that you always test skip rules before asking participants to take your survey.
We use an example to show how to use scoring rules. Suppose you're running a store and selling two categories of products, Category A and Category B. You carry different products for each category and each product has different price. For Category A, you have the following three products and prices:
- Product 1, $30
- Product 2, $20
- Product 3, $10
There are also three products for Category B:
- Product 4, $150
- Product 5, $50
- Product 6, $10
In this survey, you'd like to find out which people are willing to spend more overall in their next visits to your store. You are also interested in those people who intend to buy Product 4. The following screenshot shows the questions in the survey, and we'll allow people to make multiple selections for the Category B question to show the system's capabilities.
Now you need to add scoring rules to the questions. For the Category A question, you'd like to assign the actual price as the score for each selection, so you create the following scoring rule when a participant selects Product 1:
Similarly, you need to add rules for the other two selections. If you want to make sure there's a score for answers to a question, the scoring rules must cover all possible scenarios. In our case, the Category A question does not require a mandatory answer, so we must add another scoring rule when a participant doesn't select any product for this question. Note that if no rule applies to the answer to a question, then our system does not add any score to the total score of a response. No score for a question or response does not mean zero. It simply means that no applicable scoring rule is found.
Here are the scoring rules so far for the Category A question.
For the Category B question, you'd like to give a score larger than its price if a participant selects Product 4 (the most expensive one) and a smaller score for Product 6 (the cheapest one). Scores can be any number you deem appropriate for your survey, including negative numbers and decimals. Suppose you add the following scoring rules:
Here are the scoring rules so far on the Category B question:
For this question, the sequence of scoring rules is important because it allows for multiple selections. If a participant selects both Products 4 and 6, we'll apply the scoring rule for selecting Product 4. Once this rule is applied, the rule for Product 6 will be ignored. For the Category A question, the sequence of scoring rules has no effect because a participant can only select one product, and there is no overlap of covered scenarios between rules.
Once participants respond to your survey, you can see the score along with the answer to each question in a response when you download responses. See the following screenshot for an example. This tool can be very helpful if you use it with other analytical tools that we offer.
An Example Survey for Analysis
In the rest of the guide, we will focus on the analytical tools and use the following example survey to explain and demonstrate these tools. Suppose that you are working on a survey project with two goals:
- Understand the relationship between students' sleep and their academic performance
- See if gender or sports activities play any role
Let's assume that you have the following four questions in your survey:
Question 1 How many hours do you sleep on average, on a nightly basis? Please enter a number only.
Question 2 What is your cumulative GPA, on a 4.0 scale? Please enter a number only.
Question 3 How much time do you spend on sports on a weekly basis?
- Less than an hour
- 1 - 2 hours
- 2 - 4 hours
- 4 - 6 hours
- More than 6 hours
Question 4 What is your gender?
In this guide, you'll accomplish the following tasks:
- Create a survey with questions that collect quantitative, categorical, and ordinal data
- Conduct two-way cross tabulation between student gender and time spent playing sports
- Do a student's t test to see if GPA means are statistically different between female and male students
- See if gender and time spent playing sports are associated in any way
- Perform linear and polynomial regression between time spent sleeping and GPA
Questions and Data Types
Each question collects some data from survey participants. The type of data a question collects determines the analysis available on the question. AllCounted supports three data types:
|Categorical||Example– Hair color: Black, Brown, Blonde, etc.||Percentage, cross tabulation, Chi-square test of independence|
|Ordinal||Example – Strongly agree, Agree, Neutral, Disagree, Strongly disagree||Percentage, mean, median, mode, cross tabulation, Chi-square test of independence|
|Quantitative||Examples – Age, income, blood pressure, temperature, number of people in a family, balance on a bank account, etc.||Mean, median, mode, standard deviation, t tests, ANOVA, linear or polynomial regression|
Ordinal data is a special kind of categorical data. We make this distinction only when needed. Otherwise, they are both referred to as categorical data.
For Questions 1 and 2, you want to collect numbers. To enforce this requirement and make analysis on quantitative data possible, you have to indicate that the data type of these two questions is quantitative. Also, we highly recommend that you use validation to require survey participants to only enter numbers (with or without range requirement) because invalid data will be excluded in the analysis. See the following two screenshots:
Suppose that you select Ordinal as the data type for Question 3 and Categorical for Question 4. Please see the following two screenshots:
Analyze Your Survey
Suppose that you have collected enough responses and are ready to perform analysis. Open your survey, click Analyze, Results as shown below:
The above screenshot shows the types of responses you can use in analysis and the available analytical tools (under View). You can look at and analyze the responses to your survey in different views. The default and perhaps the most important view is Summary. In the Summary view, AllCounted lists the text answers to a Text Box question by default. If the question's data is quantitative, you'll see a Statistics link, which gives you some simple statistics on the responses to your question (Note that you can get many more statistics under the Statistics view). Please see the following two screenshot shots:
In the Summary view, for Multiple Choice questions like Question 4, AllCounted displays the statistics in a chart and table as shown below. The chart and the table have the same information, but the table is helpful if the text is too long to display on the chart.
Question 3 is special because its data is ordinal. In the Summary view, you can see more statistics such as the mean, median, and mode. Please see the following screenshot. Note that we calculate the mean for ordinal data for practical reasons. We automatically assign 1, 2, etc. to the first row (Less than an hour), second row (1 - 2 hours), etc. when calculating these statistics.
Understand Your Data First
Before you start doing advanced analysis, it's always good to have a basic understanding of the responses to the question you want to analyze. Another reason to review the responses first is because some statistical tests assume that data has met certain requirements; otherwise, an analysis could be meaningless. For example, student's t tests assume that data is normally distributed. A basic understanding of your data can help prevent this type of error.
For quantitative Text Box questions, we offer a wide range of basic statistics such as quartiles, central tendency, dispersion, and distribution. You can even split the data into a number of groups of equal percentile or find a specific percentile. In addition, you can create histograms or boxplots for quantitative responses. To do this basic analysis, click Analyze, Results, and then Exploring Quantitative Data under the Statistics view. You'll be asked to select the variable to analyze. The following two screenshots show this type of analysis on Question 2 (the GPA question).
You have four ways to do cross tabulation: by the answer choices of a Multiple Choice question, campaigns, distributions, or analysis groups. Please see the following screenshot:
For example, let's select "By a Multiple Choice Question" and click the Apply button. You'll be prompted to select a Multiple Choice question. Click the Question menu and select a Multiple Choice question. Suppose we selected Question 4 and its two answer choices, as shown below:
Click the Apply button, and you'll see the data separated by the answer choices of the selected Multiple Choice question.
The data in the above screenshot is equivalent to the following two tables (The second table is only applicable to Multiple Choice questions with the ordinal data type.):
|Less than an hour||1 - 2 hours||2 - 4 hours||4 - 6 hours||More than 6 hours|
|Female||1, 11.11%, 5.88%, 5.88%||2, 22.22%, 11.76%, 11.76%||3, 33.33%, 17.65%, 17.65%||2, 22.22%, 11.76%, 11.76%||1, 11.11%, 5.88%, 5.88%|
|Male||1, 12.50%, 5.88%, 5.88%||3, 37.50%, 17.65%, 17.65%||4, 50.00%, 23.53%, 23.53%|
|Mean, Median, Mode|
|Female||3.00, 3, 2 - 4 hours|
|Male||3.25, 3.5, 4 - 6 hours|
If the data type of a Text Box question is quantitative, then you'll see how its data differs between answer choices. The following screenshot shows how the data of Question 1 varies depending on the answer choices of Question 4.
The data in the above screenshot is equivalent to the following table:
|Mean of Sleep Hours|
Independent Samples Student's T Test
If you want to compare responses of different groups based on the same set of questions, then you need to create a single survey and collect responses through different campaigns or distributions. Each campaign or distribution counts as one group of responses. You may also separate responses into different groups based on the answer choices of a multiple choice question. In this situation, each answer choice is a label for a group of responses, and you don't need to create multiple campaigns or distributions. Note that AllCounted is unable to compare responses from two surveys even if their questions are exactly the same.
Student's t test is a commonly used tool for comparing the means from different groups of responses. AllCounted supports one sample, independent samples, and paired samples student's t tests. In this guide, we use the independent samples t test as an example. Suppose that you want to know whether the difference in GPA means between female and male students is statistically significant. Click Independent Samples Student's T Test under the Statistics view, and then click the Apply button. You'll be asked to select the variable and data groups. Here are our selections for this demo:
The following screenshot shows the results, which indicate that the difference in GPA means is not statistically significant.
Here is a neat feature for survey creators. If you'd like to export the data points in this test and put them in another analytical tool, you can click on the name of a data group to show all the data points and copy all of them with a mouse click. This feature is available for all statistical tools. The following screenshot is an example:
It is worth noting that for the two data groups in this test, you can also select two campaigns, two distributions, or two analysis groups. This applies to all the statistical tools that need two or more data groups. Besides, you can always use the Filters to further segment responses into the dataset you'd like. The possibilities are endless!
Chi-Square Test of Independence.
To conduct a Chi-square test of independence, you need to have two categorical variables. In our example, the gender and time spent playing sports questions meet this requirement. Click Chi-Square Test of Independence under the Statistics view, then click the Apply button. You'll be asked to select the variables. Here are our selections for this demo:
The following screenshot shows the results based on the above settings.
Note that in this test, you are not asked to select any data group, which means that you have only one group, which is defined by your "Filter by" settings. for this and other tests such as the one-way ANOVA we offer, you can always use filters to slice and dice data, and the results will be updated instantly.
Doing one-way ANOVA is similar to doing an independent samples t test. The only difference is that you can select only two data groups for the t test, but you can select more than two groups in the one-way ANOVA test. To do one-way ANOVA, click One-Way ANOVA under the Statistics view, then click the Apply button. You'll be prompted to select two or more data groups and the dependent variable. The following two screenshots show our selections in this demo and the results:
Note that our system is very flexible and you can select any two or more answer choices to form the data groups needed for this test.
Regression can be performed between two Text Box questions with the quantitative data type. To perform regression between Questions 1 and 2, select "Simple Linear Regression" or "Polynomial Regression" under the Statistics view and click the Apply button. You'll be prompted to select the independent and dependent variables. Let's select Question 1 as the independent variable. Click the Apply button, and you'll get the following table, function, and chart:
For polynomial regression, you can optionally specify the degree of the fitting polynomial function. You may enter a number between 1 and 20 for the degree. The higher the degree, the more time it'll take to produce the regression function and chart. If you don't select the degree, we'll use 2 by default. The following screenshot shows this:
If our analytical tools can't satisfy your needs, you can export all or some of your survey responses into a text, xml, or Excel file. Click the Results menu, select Individual Responses, and click the Apply button, which will display a list of individual responses. Then, click the Export menu and the download format. The download will start immediately. Please note that if you have a lot of responses, the download may take a while. The good news is that you can leave the download window open and start another browser window to continue using our website without waiting for the download to finish.
As mentioned above, in statistical tools, you can click the name of a data group and export all the data points used in a test.
We can help
We offer many powerful and innovative tools for online surveys, and you need to sign up to use them. All tools and services at AllCounted are completely free. If you have any questions, suggestions or see something that's unclear, missing, or incorrect, please don't hesitate to contact us at firstname.lastname@example.org. We will do our best to timely serve your needs.
If your colleagues, friends, or classmates need a tool for online surveys, please let them know that we can help. Thank you!