# Survey QA: Automated survey testing

Survey QA is a [Cassi.ai](https://www.cassiai.com) product for automated survey testing. A robot walks the survey's paths on desktop and mobile, the way a respondent would, proves every rule of the questionnaire and returns a verdict screen by screen with the screenshot and the rule for every finding. Judgment is by rule, and every finding comes with the screen that proves it.

## What it checks

- Filters and terminations: Every conditional route is walked once, with the answer that should terminate and with the one that should continue. The robot records on which screen the programming stopped.
- Consistency checks: Required fields, ranges, sums and combinations across questions. The robot tries to break each rule and records what the programming did, including the combination that only breaks on the third try.
- Quotas: Cell by cell: the closed one has to terminate, the open one has to reach a complete. When a filter sends the respondent to a block the quota has already closed, that becomes a finding with the screen and the rule.
- Order and rotation: The JobSpec list is checked item by item, in the programmed order and the expected rotation, on desktop and mobile.
- Texts and options: Question text, options and instructions compared with the document, screen by screen. The instruction inherited from the previous study shows up here, even when only a small slice of the sample would see it.
- Mobile layout: Seven checks on mobile on every screen, next to desktop: horizontal scroll, question text off screen, truncated text, overflowing grid, next button off screen, tap target and font size.
- Tracking waves: Waves compared one by one: what broke, what was fixed and what changed in the text from one wave to the next.
- Screen statement: Every session records the screens it walked, the route, the objective and the screenshot of every screen. The statement is readable by the client without a translator, and no route is repeated.

## How the test is done

1. Compile the questionnaire, the JobSpec and the script into a single truth. Word, Excel, MDD or a Dimensions script become one contract: questions, options, rules, brands and attributes. A conflict between sources blocks approval until the corrected document is uploaded.
2. Teach the robot with a person. Someone answers the survey once, with desktop and mobile side by side. The recording teaches the path, the selectors and the way each platform advances.
3. Plan the routes, one per objective. The plan comes from the contract: every filter, quota, consistency check and list gets its own route. Repeated routes are forbidden, and the person approves the plan before running.
4. Walk it on desktop and mobile. The robot navigates every route at both sizes at the same time, screen by screen, with a screenshot and a measurement at every step.
5. Judge screen by screen, by rule. Every rule in the contract becomes a check: the robot tries to break the rule and records what the programming did. The verdict comes from rules, with no language model.
6. Return the verdict screen by screen. One sentence, four numbers and the screen-by-screen ledger with the screenshot of every screen open. A client link in three languages, a one-page PDF and a QCF spreadsheet per screen and per variable.

## Proof

- A filter that closes on its own: The profile question sends the respondent to a block the quota has already closed. They see no error. They see the end of the questionnaire, and enter the database as a dropout.
- A consistency check that only breaks on the third combination: Age, region and purchase frequency pass separately. Together, they send the respondent to another target's block. Manual testing rarely reaches that combination.
- A return route that erases the open answer: The respondent gets the format wrong, corrects it, and lands on the screen before the one they were on. Fieldwork goes on, and the open answer from that screen is gone.
- Text inherited from the previous study: The instruction of an open question kept the text from the previous study. It only appears to whoever falls into that filter, always a small slice of the sample.

- 7 layout checks on every mobile screen (rule of every session)
- 2 sizes per screen: desktop and mobile (rule of every session)
- 0 AI calls in judgment (rule of the method)
- 1 route per objective, none repeated (rule for every session)

Platforms: WebRaptor, Sawtooth, IBM Dimensions, Qualtrics, Confirmit/Decipher/Forsta, Alchemer and QuestionPro. The robot recognizes the platform on the first screen and discovers how to advance.

## What it never does

- Never approves the release to field in your place. Your analyst still signs.
- Never forces a step when the questionnaire refuses to advance. The refusal becomes a finding, with the screen.
- Never uses a client's questionnaire, answers or screenshots without authorization.
- Never shows "passed" without the list of what was verified.
- Never claims coverage it did not measure. Every coverage sentence states its scope.
- Never works around anti-bot barriers or CAPTCHA. It detects, reports and calls a person.
- Never repeats a route already charged. Every session has its own objective.

## Who it is for

Research firms, Fieldwork agencies, Insights teams, Questionnaire programmers.

## Questions and answers

### What is Survey QA?

Survey QA is a Cassi.ai product for automated survey testing. A robot walks the survey's paths on desktop and mobile, the way a respondent would, and returns a verdict screen by screen with the screenshot and the rule for every finding.

### What does the test check?

Filters and terminations, consistency checks across questions, quotas, order and rotation of brands and attributes, texts and options against the approved questionnaire, mobile layout in seven checks and the comparison between waves of a tracking study.

### Which platforms does it work on?

WebRaptor, Sawtooth, IBM Dimensions, Qualtrics, Confirmit/Decipher/Forsta, Alchemer and QuestionPro. The robot recognizes the platform on the first screen and discovers how to advance. Your platform goes through the same path: send the link.

### Does the robot use artificial intelligence?

Not in judgment. Every rule of the questionnaire becomes a deterministic check, and the verdict comes from rules, with zero AI calls. The reading of the documents is checked by a person before the robot runs.

### What does the client receive at the end?

One screen with the verdict in one sentence, four numbers and the screen-by-screen ledger with the screenshot of every screen open. Along with it go the client link in Portuguese, English and Spanish, a one-page PDF and the QCF spreadsheet per screen and per variable.

### How much of the questionnaire is covered?

What the approved plan enumerates, and only that. Every coverage sentence states its scope: routes planned, routes walked, screens reached. The robot never claims coverage it did not measure.

### What happens when the questionnaire has a CAPTCHA or an anti-bot barrier?

The robot detects it, reports it and stops. It never works around the barrier. The finding comes out with the screen, and a person decides how to proceed, usually with the test link without the barrier.

### Who approves the release to field?

Your analyst. Survey QA delivers the list of what was verified, with the screenshot of every screen, and never approves fieldwork in place of whoever signs.

### How do I get access?

Through the access request on this page. Products available via SaaS by credit, by project, or as a platform purchase. Talk to us, we have a format that fits your budget.

## How to get access

Access is restricted to clients with an active project. Requests go through the form at https://surveyqa.cassiai.com/. Products available via SaaS by credit, by project, or as a platform purchase. Talk to us, we have a format that fits your budget.

Article on the Cassi.ai blog: https://www.cassiai.com/blog/survey-testing-before-fieldwork