Refinement workflow

After a question is coded, you almost always want a second pass. Refinement is Survey Coder Pro’s AI-assisted second pass: an agent reviews the codebook + the coded results and proposes targeted edits — new codes for groups of unclassified responses, tighter definitions for codes that drift, splits for codes that lump two things. The whole workflow is exposed via the REST API and both SDKs, so you can drive it from your own scripts without opening the dashboard.

Note

Refinement only runs against questions that have already been coded (you have a coded_responses table populated for the question). If you call it on a freshly-uploaded question, the agent has nothing to refine.

How the agent works

The agent (Claude) receives:

1. The full codebook — every category + code + definition + examples.
2. The unclassified responses — verbatims with no row in coded_responses.
3. The low-confidence responses — verbatims the coding pipeline assigned with confidence: 'low' (or medium if your question’s review_threshold is set higher).
4. The original question_text — what was actually asked.
5. Your coding_guidance — any free-form instructions you passed at coding time.

It processes the inputs in batches of 50 and emits structured suggestions. Each suggestion has one of three types:

Type	What it proposes
new_code	Create a new code for a group of unclassified verbatims that share a theme
update_definition	Tighten the definition of an existing code to reduce false positives/negatives
split_code	Divide a code that’s lumping two distinct themes

Plus a confidence (0..1) the agent self-reports, a list of example_responses, and reasoning (plain text) explaining why.

Caching

The agent run is cached — once a question has pending suggestions in the DB, calling /v2/refinement/suggestions again returns them without re-running the AI (saves credits). To force a fresh run (which drops any pending suggestions and re-charges credits), pass force: true.

Endpoints

Endpoint	What it does
POST /v2/refinement/suggestions	List or generate the agent’s suggestions
POST /v2/refinement/apply	Apply one suggestion (creates the code, reassigns responses)
POST /v2/refinement/resolve	Mark a suggestion as resolved with a specific action — create_new, assign_existing, merge, or dismiss
POST /v2/refinement/undo	Reopen a previously-resolved suggestion (does NOT revert mutations — treat as “reopen”)
POST /v2/refinement/bulk-apply	Apply many new_code suggestions in one call, optionally filtered by min_confidence
POST /v2/refinement/export	Download every suggestion (pending + applied + dismissed) as Excel or CSV

A typical scripted workflow

The killer use case for the API is “auto-apply everything the agent is confident about, dismiss the rest, hand the analyst an Excel of what you did”.

Python

from surveycoder import SurveyCoderClient
from pathlib import Path

client = SurveyCoderClient(api_key=os.environ["SCP_API_KEY"])

PROJECT_ID = "ab91e16b-..."
QUESTION_ID = "d8cfd844-..."

# 1. List current suggestions
result = client.refinement_suggestions(PROJECT_ID, QUESTION_ID)
suggestions = result["suggestions"]
print(f"{len(suggestions)} suggestions pending")

# 2. Auto-apply the high-confidence new_code suggestions
applied = client.bulk_apply_refinement(
    PROJECT_ID, QUESTION_ID,
    min_confidence=0.75,
)
print(applied["summary"])
# {'candidates': 12, 'applied': 7, 'skipped': 5, 'failed': 0}

# 3. Export the full audit trail to Excel for the analyst
xlsx_bytes = client.export_refinement(PROJECT_ID, QUESTION_ID, format="xlsx")
Path("refinement_report.xlsx").write_bytes(xlsx_bytes)
print("Saved refinement_report.xlsx")

TypeScript

import { SurveyCoderClient } from 'surveycoder-sdk';
import { writeFileSync } from 'node:fs';

const client = new SurveyCoderClient({ apiKey: process.env.SCP_API_KEY! });

const PROJECT_ID = 'ab91e16b-...';
const QUESTION_ID = 'd8cfd844-...';

// 1. List
const { suggestions } = await client.refinement.suggestions({
  project_id: PROJECT_ID,
  question_id: QUESTION_ID,
});
console.log(`${suggestions.length} suggestions pending`);

// 2. Auto-apply high-confidence new_code
const applied = await client.refinement.bulkApply(PROJECT_ID, QUESTION_ID, {
  minConfidence: 0.75,
});
console.log(applied.summary);

// 3. Export to Excel
const xlsx = await client.refinement.export(PROJECT_ID, QUESTION_ID, 'xlsx');
writeFileSync('refinement_report.xlsx', xlsx);

curl

# 1. List
curl -X POST https://api.surveycoder.io/v2/refinement/suggestions \
  -H "x-api-key: $SCP_API_KEY" -H "Content-Type: application/json" \
  -d '{"project_id":"ab91e16b-...","question_id":"d8cfd844-..."}'

# 2. Bulk apply (only `new_code` suggestions with confidence >= 0.75)
curl -X POST https://api.surveycoder.io/v2/refinement/bulk-apply \
  -H "x-api-key: $SCP_API_KEY" -H "Content-Type: application/json" \
  -d '{
    "project_id":"ab91e16b-...",
    "question_id":"d8cfd844-...",
    "min_confidence":0.75
  }'

# 3. Export to xlsx
curl -X POST https://api.surveycoder.io/v2/refinement/export \
  -H "x-api-key: $SCP_API_KEY" -H "Content-Type: application/json" \
  -d '{"project_id":"ab91e16b-...","question_id":"d8cfd844-...","format":"xlsx"}' \
  -o refinement_report.xlsx

What bulk_apply does and doesn’t do

It only applies suggestions of type new_code. The other two types need decisions the API can’t make blindly:

• update_definition — changing a code’s wording can break downstream analyses; the analyst should review.
• split_code — splitting a code requires deciding which responses go on each side; bulk can’t infer that.

For those, use POST /v2/refinement/resolve with the right action per suggestion, or build a UI flow on top of the suggestion list.

Suggestions below your min_confidence threshold are listed in the skipped array with a reason like "confidence 0.62 < min_confidence 0.75".

The response shape:

{
  "applied": ["sugg_abc", "sugg_def"],
  "skipped": [
    { "id": "sugg_xyz", "reason": "confidence 0.62 < min_confidence 0.75" },
    { "id": "sugg_qqq", "reason": "type='update_definition' is not bulk-appliable (only 'new_code' supports create_new without overrides)" }
  ],
  "failed": [],
  "summary": { "candidates": 4, "applied": 2, "skipped": 2, "failed": 0 }
}

What export returns

The Excel (xlsx) has four sheets:

Sheet	Rows
Summary	High-level counts: total / pending / applied / dismissed, plus the question and project IDs and export timestamp
Pending	Every suggestion not yet resolved. Columns: id, type, category, code_name, definition, affected_count, confidence, reasoning, examples (up to 3 verbatims resolved server-side), created_at
Applied	Suggestions resolved with a non-dismiss action. Same columns + resolved_at, resolved_action
Dismissed	Suggestions resolved with dismiss. Same columns + dismiss_reason if one was provided

The CSV (csv) is a single flat table with the same data and an extra status column (pending / applied / dismissed). Use it when you want to grep or pipe into pandas / a SQL load.

Tip

The Excel example texts are resolved server-side — the file is self-contained, you don’t need to separately export the verbatims. Useful when handing the file to a client who doesn’t have dashboard access.

Refinement vs Coding — credit cost

• The first refinement_suggestions call on a question consumes credits for the LLM pass (variable based on the number of unclassified + low-confidence responses).
• Subsequent calls return the cached suggestions for free.
• force: true re-runs and re-charges.
• apply, resolve, undo, bulk_apply don’t call the LLM — they just mutate Postgres rows. No credit cost.
• export is also LLM-free.

What undo does NOT do

POST /v2/refinement/undo is a misnomer in the current implementation — it only sets the suggestion’s status back to pending. It does not revert the codebook/coding mutations the prior apply produced. A true revert requires snapshotting the pre-apply state (planned, not shipped). Today: treat undo as “reopen this for re-evaluation”, and if you need a real revert, do it manually via the dashboard’s codebook editor.

Related

• Coding pipeline — what produces the codes refinement operates on
• SDK reference: Python · TypeScript
• API reference