1. Why AI Annotation Pricing Is So Opaque
If you've tried to budget an annotation project in the last 12 months, you already know the problem: no vendor publishes rates. Scale AI says “contact us.” Appen says “request a quote.” Labelbox publishes platform subscription tiers but nothing about workforce cost. The result is that ML teams enter procurement cycles with no price anchor — which means the vendor controls the negotiation from the first call.
The opacity is not accidental. Annotation vendors benefit from information asymmetry. When you don't know that 1,000 preference pairs from domain experts costs $2,000–$8,000 at market rates, you have no way to evaluate whether a $15,000 quote is reasonable, inflated, or a loss-leader. “Contact us” pricing exists to prevent exactly the comparison this post makes possible.
The second reason is that annotation cost genuinely varies — by task type, annotator tier, quality methodology, and volume. But “it depends” is not a useful answer when you're writing a Q3 budget. This post acknowledges where real uncertainty exists and anchors every range. By the end, you should have enough to write a first-pass budget request — or at least call a vendor's first quote.
2. The Four Annotation Pricing Models
Before diving into dollar ranges, it helps to understand the four ways annotation is priced — because the model affects both total cost and what risks you're absorbing.
Per-label / per-annotation
The commodity crowdsourcing model. You pay a fixed amount per completed annotation task. Typical ranges: image bounding boxes and classification tasks run $0.01–$0.05/label at scale. Crowdsourced NLP annotation (sentiment, entity tagging, basic text classification) runs $0.05–$0.25/annotation. This model is transparent and predictable. The risk: per-label pricing incentivizes throughput over quality. Annotators are paid to complete tasks, not to be accurate — which is acceptable for commodity tasks and problematic for anything requiring judgment.
Per-hour
The model used by specialist and domain expert annotators. You pay for annotator time, not for completed tasks. Typical ranges: $15–$25/hr for general-purpose workers with tested skills, $35–$60/hr for domain specialists with verified credentials (MDs, JDs, engineers), $60–$85/hr for senior domain experts or rare specializations. Per-hour pricing is common for RLHF annotation, medical coding, legal review, and complex model evaluation tasks where task completion rate is a poor quality signal. Risk: you absorb scope uncertainty — if the task takes longer than estimated, cost increases.
Per-project
Fixed-scope dataset packages with a defined deliverable. You pay a flat amount for a specified number of annotations, annotator tier, and quality methodology. Human Consensus AI uses this model — $49 for the Starter Pack (25–50 expert preference pairs), $299 for the Enterprise Bundle. Per-project pricing gives you cost certainty and aligns vendor incentive with deliverable quality rather than time or volume. Risk: scope creep if your requirements change mid-project; fixed-scope packages work best when the task is well-defined before you buy.
Subscription / platform seat
The Labelbox and SuperAnnotate model. You pay for access to labeling infrastructure and tooling on a recurring basis — but the annotator workforce is separate. Labelbox platform pricing runs $1,800–$4,500/month depending on features and seat count. This does not include annotator wages: you supply your own annotators, hire contractors, or source from Labelbox's marketplace. Subscription pricing works for teams with existing annotator pools who need enterprise tooling. For teams without annotators, the platform cost is a layer on top of, not instead of, workforce cost.
3. What Drives Annotation Cost — The 5 Variables
Two annotation projects with identical task counts can differ 10× in cost. These five variables explain most of that spread.
1. Annotator expertise level
The single biggest cost driver. Generalist crowdworkers cost $0.02–$0.15/annotation. Domain experts — credentialed specialists with verified background in your vertical — cost $0.50–$2.00/annotation or more. The difference is 3–10×. For commodity tasks (image classification, basic NLP), generalists are appropriate. For RLHF preference annotation, medical AI evaluation, code generation assessment, or legal document AI, domain expertise is not optional — it's the quality mechanism. See the full breakdown in our guide on domain expert annotators vs. crowdsourcing.
2. Task complexity
Image classification (yes/no, multi-class) is the cheapest annotation task per item. Bounding box annotation is more expensive. Named entity recognition adds another step. RLHF preference annotation — where annotators must evaluate multiple model outputs on several quality dimensions and provide a ranked preference with rationale — is 5–20× more expensive per item than simple classification. Medical coding and legal document annotation sit at the top of the complexity range. Complexity affects both the time per task and the annotator tier required.
3. IAA / quality requirements
How you measure and enforce quality has a significant effect on cost. Majority vote across 3 annotators is cheaper than consensus with documented rationale across 3–5 annotators. Requiring inter-annotator agreement (IAA) documentation at the κ ≥ 0.70 level typically means 2–4× more annotator time per item compared to single-annotator labeling. This is often the right tradeoff: noisy preference data used in a training run costs far more to fix than the IAA overhead costs to prevent.
4. Volume
Most vendors offer volume discounts above 5,000–10,000 annotation tasks. Per-unit cost typically drops 20–40% when moving from a 1,000-annotation pilot to a 10,000-annotation production run. At 50K+ annotations, enterprise pricing applies — and scale becomes a genuine negotiation lever. If you're planning a large run, price both a pilot and the production volume to understand the per-unit economics at scale.
5. Turnaround time
Rush premiums are common, particularly for expert annotators. Standard turnaround for a 1,000-preference-pair dataset with domain experts is 7–14 business days. Expedited delivery (3–5 days) typically adds 25–50% to cost. For crowdsourced tasks at scale, turnaround is faster and rush premiums are smaller. If your model launch is in two weeks, factor turnaround explicitly into platform selection — not just pricing.
4. Real-World RLHF Dataset Budget Benchmarks
These are market-rate ranges for common RLHF and SFT annotation use cases, based on publicly available information and vendor conversations. They are not Human Consensus AI prices specifically — we'll be explicit about where our products sit at the end.
| Use Case | Crowdsourced Range | Domain Expert Range | Notes |
|---|---|---|---|
| 1,000 preference pairs (RLHF pilot) | $500–$2,500 | $2,000–$8,000 | Includes 3-annotator consensus; no IAA docs at low end |
| 10,000 preference pairs (RLHF production) | $3,000–$15,000 | $15,000–$60,000 | Volume discount applies; wide range reflects task complexity |
| 50K+ preference pairs (enterprise RLHF) | $30,000–$100,000+ | $50,000–$300,000+ | Scale AI / enterprise territory; negotiated contract pricing |
| SFT instruction dataset (10K examples) | $2,000–$10,000 | $8,000–$30,000 | Writing quality and domain depth drive expert cost |
To be direct about Human Consensus AI pricing: the $49 Starter Pack and $299 Enterprise Bundle are scoped pilot datasets — structured samples designed for teams validating annotation quality before committing to a production run. They are not full-scale production RLHF datasets. If you need 10,000+ preference pairs, you're looking at the production ranges above.
For a deeper look at how annotation volume affects per-unit economics, see our guide on scaling RLHF to 10,000 annotations.
5. The Hidden Costs Teams Forget
The annotation invoice is not the total cost of a dataset. Four cost categories consistently show up in post-mortems that were absent from the original budget.
Rework cost from low-quality data
A low-quality annotation dataset used in a training run, discovered after the fact, produces a retraining cost that typically dwarfs the original annotation savings. Concrete example: 5% label noise in a 50,000-preference-pair dataset cascades into reward model drift that's not detected until eval. Correcting it requires identifying the noisy subset, re-annotating, and retraining — often 2–3 weeks of engineering time plus $10,000–$50,000 in GPU compute. Saving $3,000 on cheap crowdsourced annotation and absorbing a $30,000 retraining cost is a common and avoidable outcome.
Quality validation overhead
If you don't pay for IAA methodology and consensus documentation up front, you're paying internal staff to audit after delivery. A typical audit cycle on a 1,000-annotation dataset — spot-checking, flagging disagreements, documenting reliability — takes an ML engineer 4–8 hours. At fully-loaded cost, that's $400–$1,200 of internal labor not in the annotation line item. For teams running multiple annotation batches, this overhead compounds. See our domain expert vs. crowdsourcing comparison for the IAA data behind this.
Platform seats on BYO tools
Labelbox, SuperAnnotate, and similar platforms charge for platform access separately from annotator wages. If you're managing your own contractor pool through a labeling tool, add $1,800–$4,500/month in platform cost to your annotation labor cost. Teams that don't account for this in the initial budget often discover it late — when the platform invoice arrives in month two of a project that was scoped to fit in month one.
Coordination overhead
Managing your own annotator pool — recruiting, onboarding, calibrating, monitoring quality, resolving disputes — is real work. For a typical ML team running a self-managed annotation program, coordination overhead runs 15–25% of total project time. On a two-month annotation project with a two-person ML team, that's roughly 40–80 hours of internal labor. Budget for it or buy it out of scope with a full-service provider.
Want a pilot dataset with transparent pricing and IAA documentation?
The Human Consensus AI Starter Pack gives you 25–50 expert-annotated preference pairs with κ ≥ 0.70 IAA documentation — $49, no contract, delivered this week.
Get the Starter Pack — $49 →6. How to Build an Annotation Budget — Practical Framework
Here's a five-step framework for building a defensible annotation budget. It works for RLHF datasets, SFT instruction data, and model evaluation annotation.
Step 1: Define the task type and IAA requirement
What are annotators actually doing — preference ranking, instruction writing, domain question answering, bounding boxes? And what inter-annotator agreement level does the task require? RLHF preference annotation for reward model training typically requires κ ≥ 0.60–0.70. Image classification for basic CV tasks may be acceptable at κ ≥ 0.45 with statistical filtering. Task type determines annotator tier; IAA requirement determines annotator count per item.
Step 2: Estimate annotation count with overhead
Your training set size is not your annotation count. Add: 10–15% calibration set (annotated twice to measure rubric consistency), 5–10% QA buffer (items flagged for re-annotation), and 5% pilot batch (for testing your rubric before the full run). Total overhead typically runs 20–30% above the raw training set count. A 5,000-preference-pair training set requires roughly 6,000–6,500 annotation tasks to deliver.
Step 3: Price the annotator tier
Use the market benchmarks from Section 4 as your anchor. For crowdsourced annotation: $0.05–$0.25/annotation for NLP tasks. For domain expert annotation: $0.50–$2.00/annotation depending on expertise level and task complexity. For consensus-based annotation with documented rationale (3 annotators, κ ≥ 0.70): multiply the per-annotation rate by 3–4× to account for the multi-annotator methodology.
Step 4: Add platform / tooling cost if BYO
If you're using Labelbox, SuperAnnotate, or similar: add $1,800–$4,500/month for platform access. If using a full-service provider (Scale AI, Human Consensus AI): platform cost is included in the per-annotation price. If self-hosting annotation infrastructure: add engineering setup cost — typically 20–40 hours one-time.
Step 5: Add rework buffer
Add 10–20% to your total estimated cost as a rework reserve. This is not pessimism — it's accounting for the real cost of edge cases, rubric revisions, and re-annotation requests that arise on nearly every annotation project. Teams that skip this buffer tend to come back with a change order mid-project.
Worked example: 5,000 preference pairs, domain experts, IAA ≥ 0.70
For how these numbers shift at higher volume (10,000+ preference pairs), see the scaling RLHF to 10,000 annotations guide.
7. Where Human Consensus AI Fits in the Pricing Landscape
Honest positioning, because this is the last thing a buyer should read before making a decision.
The $49 Starter Pack and $299 Enterprise Bundle are scoped pilot datasets. They're priced for teams that need structured, expert-annotated samples with IAA documentation — not for teams building 50,000-pair production RLHF datasets. That distinction matters, and we'd rather be clear about it than oversell scope.
Right for Human Consensus AI if:
- ✓Validating whether expert annotation moves your eval metrics before committing to a $15K–$60K production run
- ✓Needing a fast-turnaround calibration dataset to benchmark annotator agreement on your task rubric
- ✓Early-stage AI company with limited annotation budget that needs to demonstrate a credible annotation methodology to investors or customers
- ✓Teams that need domain experts (MDs, engineers, legal professionals) without a six-figure vendor contract
- ✓Research labs or academic teams building evaluation benchmarks on a project budget
NOT right for Human Consensus AI if:
- ✗You need 50,000+ preference pairs for a production RLHF program — that's Scale AI or enterprise territory
- ✗Commodity image classification or bounding box annotation at 100K+ volume — crowdsourcing platforms win on unit economics here
- ✗You need deep integrations with enterprise ML infrastructure (MLflow, SageMaker Ground Truth, Vertex AI)
- ✗You need SOC 2 Type II or HIPAA-covered annotation at enterprise scale
How we compare on price:
For teams that have already done the vendor comparison research — read the Scale AI alternatives comparison, the Labelbox alternatives guide, and the Appen alternatives breakdown — the question is no longer which vendor but whether the budget fits the scope. If you're validating expert annotation ROI before a production commitment, the Starter Pack is the right entry point. If you're scoping a production annotation program and need help choosing the right tier and vendor, the AI training data partner selection guide covers the full decision framework.
25–50 expert-annotated preference pairs, κ ≥ 0.70 IAA documentation, RLHF-ready format. No contract, no onboarding call, delivered this week. Right for teams validating expert annotation methodology before a larger commitment.
Larger annotation volume, custom rubric design, dedicated domain expert sourcing, and ongoing IAA monitoring. For teams building a production-quality annotation foundation without a six-figure vendor contract minimum.