AI Decision APIs

Executive Summary

An AI decision API turns application context into a value software can use: a class, a rubric score, or a probability. The important distinction is how that judgment is defined and maintained. TypeSafe/Jev evaluates developer-defined questions; Nyckel manages classification functions that improve from annotations; Amazon Comprehend trains and serves explicitly versioned document classifiers. ^[1][2][3][4]

These 3 selected providers illustrate different approaches to configurable decisions. None is an evidence-backed universal winner. Compare them at a useful error rate and review workload, with the required modality, release controls, and data terms. Public documentation and selected first-hand reports were checked on September 16, 2026; this site did not run a common benchmark or purchase service capacity.

---

Scope and Candidate Discovery

Membership requires an externally callable hosted product that accepts application context and supports customer-defined labels or judgment criteria. This report focuses on those configurable workflows for new pilots. It separates general prompt-defined decision models from managed trained classifiers. An early-access product can be evaluated here without implying unrestricted account availability.

Task-specific APIs form an adjacent category. Google Cloud Natural Language's classifyText, for example, returns categories from Google's predefined taxonomy. That can be sufficient for content organization, but is different from defining an organization's own ticket labels or scoring rubric. It is boundary context rather than a fourth comparison member. ^[5]

Discovery also checked two historically relevant custom-classification offerings:

• Cohere Classify: Cohere's September 2025 notice marks /v1/classify deprecated and retires classification fine-tuning among other capabilities. It is not included as a current new-project recommendation. ^[6]
• Azure Language custom text classification: Microsoft still documents the service, but now announces retirement on March 31, 2029 and directs all new projects to Microsoft Foundry. Existing users have a migration decision; the service has not already shut down. This report does not treat the entire Foundry platform as a like-for-like classifier endpoint. ^[7]

Inference hosting, agent orchestration, workflow builders, and structured-output libraries are excluded unless the product supplies the decision service itself. An LLM implementation with structured outputs is an important technical baseline, not automatically another member. The shortlist is deliberately selected, not an exhaustive census of moderation, fraud, language, vision, or industry-specific APIs.

---

Comparison Matrix

TypeSafe/Jev: Questions Over Shared Context

Jev fits a workflow with several narrow judgments over the same application state. For a support case, one request could ask which queue applies, how severe the issue is on a defined rubric, and whether the customer explicitly requested a refund. Those outputs have different meanings: Choice selects among named options; Score is a probability-weighted position across ordered rubric levels; Noul estimates the probability of a yes answer. ^[8]

Questions in one request are independent. If a later decision needs a database lookup determined by the first answer, application code must make that lookup and issue a subsequent request. The documented combined budget for state and questions is around 32,000 tokens. The reviewed interface supports text and structured state; the launch's structured-state game demo is not evidence of image understanding. ^[9][10]

The September 15 launch describes waitlist-based early access. Public documentation uses jev-latest; a production model-pinning contract or published numeric account quota was not verified in this review. Account access, repeatability across upgrades, and capacity therefore belong in a pilot's acceptance criteria. ^[10][8]

Nyckel: A Managed Learning Loop

Nyckel's classification surface handles text, image, or tabular input and returns a label plus confidence. Its zero-shot baseline allows initial predictions; labeled samples then support a private trained model. Nyckel selects architectures and promotes validated replacements automatically, while the current active model continues serving during retraining. Custom training and private inference appear in the Starter plan and above. ^[11][2][13]

This favors a recurring category decision whose quality improves from domain-expert corrections. The interface is stable while the implementation can change. That is convenient for an evolving classifier but different from approving a particular model release before it serves production. Prior versions are available in the console; this review did not verify a public inference-version pin or portable model export. ^[11]

The training guide documents cross-validation for its accuracy estimate and a 512-token text/tabular model context. Long tickets or documents need an explicit preprocessing strategy. Preserve decisive information when shortening or splitting; adding an image modality does not remove the documented text constraint. ^[14]

Amazon Comprehend: A Trained Document Taxonomy

Comprehend custom classification requires labeled examples. Plain-text models support six documented languages, one per model; native-document training uses English and can incorporate text/layout signals. A scanned form image is therefore a different problem from Nyckel's general image-classification function. ^[3]

Its CSV minima are 50 examples per class for multi-class training, or at least 10 per class and 50 overall for multi-label training. Synchronous plain text is limited to 10 KB; supported semi-structured synchronous input is limited to one page and 10,000 characters, alongside file-size limits. Larger document workflows should evaluate the asynchronous path. ^[15]

Named model versions and explicit endpoint updates provide a deliberate release step. Plain-text flywheels help organize retraining, but their label sets are fixed: additional labels in later datasets are ignored. Treat changing the taxonomy as a model-design change rather than assuming feedback alone adds classes. ^[4][16][17]

Custom classification remains available. The April 30, 2026 new-customer restriction covers topic modeling, event detection, and prompt safety classification, with qualifying existing customers retaining access. It does not close the entire Comprehend service. ^[18]

---

Confidence, Calibration, and Review

Correct output shape and correct judgment are separate properties. Calibration asks whether stated probabilities correspond to observed correctness across many examples. Research by Guo and colleagues establishes why calibration deserves separate measurement from classifier accuracy; it does not evaluate these three current services. ^[19]

Do not port a threshold of 0.9 between providers merely because each exposes a number. Even within Jev, the probability of a proposition being true differs from certainty about a chosen category. A low probability of “refund requested” is a likely no; it is not automatically a reason for review. ^[8][20]

For a useful evaluation, collect representative cases with independently adjudicated outcomes, separate development data from a final holdout, and measure each candidate's behavior at comparable operating points. Report per-label precision/recall, accepted-case error rate, coverage, and the review fraction. Check ambiguous and out-of-taxonomy cases explicitly. Recheck after model changes or shifts in traffic. These are evaluation recommendations, not measurements performed for this report.

A Concrete Routing Workflow

Consider routing customer tickets to billing, engineering, or general support. Define an “other/unclear” path and a separate criterion for cases requiring a person. Then:

1. Prepare context: keep the relevant customer message and authorized case information. Record preprocessing and schema versions.
2. Ask for the judgment: use Jev's defined options, a Nyckel classification function, or a trained Comprehend taxonomy. Extra rubric questions are native to Jev; they should not be presumed equivalent to a single classifier label.
3. Validate the response: handle service errors, unknown values, missing fields, and document warnings before business logic runs. ^[8][12]
4. Apply measured policy: route only the accepted cases automatically. Send uncertain or unsupported cases to review. Authorization to refund, change an account, or modify code remains a separate rule.
5. Record the decision: retain the input identifier, relevant output, provider/function/model information available, policy version, action, and eventual adjudicated result.
6. Learn without losing the audit: send corrections into the supported training workflow and reassess thresholds after changes.

Nyckel's invoke capture is useful for feedback, but selective: it prioritizes informative samples and applies deduplication and caps. It is not a complete transaction log. Its documentation explicitly recommends keeping the full prediction record in the application, keyed with externalId. Comprehend flywheels similarly manage training data rather than implementing the surrounding business approval policy. ^[24][25][17]

---

Latency and Load

These figures are not a shared benchmark. Measure total application time, including preprocessing, network travel, queueing, retries, and downstream checks. Define an error or review path when the synchronous budget expires rather than interpreting a late answer as permission to act.

The independent evidence is still limited. Every's head of evals, Mike Taylor, reported 777 Jev judgments across 37 documents in under 0.7 seconds on September 15, while calling for a stronger accuracy check before production. The same article describes Dan Shipper's 12-passage synthetic test: Jev was faster, but caught six of seven intended defects compared with seven for Fable 5.1. This is useful early evidence of a tradeoff, not a production ranking. ^[27]

TypeSafe's own evaluation also needs careful interpretation: it averages four workflows and compares with reference answers generated by larger models. Agreement with those references is not independently established correctness on real business outcomes. Neither that harness nor the Every experiments compare Nyckel and Comprehend against Jev on the same labeled workload. ^[28][27]

---

Pricing and Cost per Correct Decision

Public prices checked September 16, 2026 use fundamentally different meters:

For arithmetic context—not a quality-adjusted ranking—one million Jev calls averaging 1,000 billed input tokens cost $42. One million monthly Nyckel invokes on Business cost $1,299 before extra samples or other charges. One continuously running Comprehend IU costs $1,296 over 30 days, but that capacity does not imply it can process the same million inputs. Character lengths, arrival patterns, and supported task shapes decide whether the comparison is meaningful. ^{[10][13][29][15]}

A useful operating measure is:

Cost per correct automated decision =
  (service + training + labeling + review costs)
  / correct decisions completed automatically

Also report total resolved cases and the cost/consequences of mistakes. Otherwise, a system can appear cheap by reviewing almost nothing while making unacceptable errors, or appear accurate by automating almost nothing. For illustration, accepting 8,000 of 10,000 cases at 98% accepted-case accuracy means 7,840 correct automated decisions, 160 wrong ones, and 2,000 cases needing another path. Those are hypothetical counts, not measured product results.

---

Model Changes, Data, and Hosting

For Nyckel, capture=false opts an invoke out of its training/review capture queue. It is not a verified blanket no-retention guarantee. Its terms reserve third-party AI evaluation during training while restricting customer training data to that customer's models; “private inference” should not be stretched into “no third party ever processes data.” Enterprise pricing lists dedicated infrastructure and regional deployment, which is not proof of customer-operated model hosting. ^[24][31][13]

For Comprehend, the general FAQ permits service-improvement use and describes possible cross-region storage for that purpose. AWS Organizations opt-out policies can prevent improvement use and delete historical improvement copies, while preserving content required to deliver the service. PrivateLink does not change those terms by itself. ^[33][34][32]

If reproducibility matters, record both the API contract and the model/release information available. A stable URL or JSON schema is not proof that the classifier underneath is unchanged. A changed taxonomy, prompt, preprocessing rule, trained model, or action threshold warrants a new evaluation.

---

Tembo as Adjacent Execution Infrastructure

Tembo runs coding agents with repository and ticket context, cloud execution environments, and output for team review. It fits the execution stage around a decision service rather than membership as a classifier provider. ^[35]

A concrete application-designed pattern is to classify incoming engineering requests, send unsupported cases to a person, and use the accepted route to prepare an approved agent task. After execution, another check could flag whether the result addresses the original request. Neither classifier confidence nor that second check replaces code tests, access controls, or human approval where required. No native Tembo integration with these three APIs was verified. Disclosure: Ry Walker is Tembo's CEO and co-founder.

---

Choosing a Pilot

• Evaluate TypeSafe/Jev when the task is several narrow judgments over shared text/structured context and early access is available. Start by proving the rubric and accepted-case quality, then verify operating limits. ^[9][10]
• Evaluate Nyckel when domain experts can correct recurring classifications and managed model improvement is desirable. Validate modality/context constraints, capture policy, and automatic update behavior. ^[2][14][24]
• Evaluate Comprehend when labeled document categories, explicit model releases, and an AWS operating environment fit the problem. Model the difference between batch inference and continuously provisioned endpoints. ^[3][4][29]

Use the same held-out cases, allowed error budget, human-review capacity, and production traffic assumptions. Keep a simpler rules-based or structured-output baseline wherever it can answer the same question. The decision to automate should follow measured outcomes at the chosen operating point, not the name of the model family or the presence of a confidence field.

---

Research by Ry Walker Research • methodology