Your output is only as good as the signal you work with.
A controlled output quality test: same AI model, same questions, same budget.
0 / 10
Questions won by Amdahl
0.0
Amdahl avg. score (of 5)
0.0
Standard connectors score
+0%
Score improvement
What was tested.
The same AI model (Claude Opus) answered 10 real GTM questions spanning win/loss, competitive intelligence, pipeline health, and voice-of-customer research. Every question ran twice, once down each path.
Scoring breakdown by dimension.
An independent AI judge scored every answer from 0 (unusable) to 5 (excellent) on each dimension below. The weights blend those into a single overall quality score.
Output with Amdahl vs. standard connectors.
Per-question quality scores from the same judge, same 0 to 5 scale.
Standard connectors edge ahead only on the two pipeline questions at right, where raw CRM data was enough.
Why the gap exists.
Piped raw into an AI, call recordings and CRM exports burn the context window on grunt work: figuring out which field means what, skimming hundreds of transcripts for the three that matter. By the time real analysis starts, the window is nearly spent, and in several runs the model filled the gaps with fabricated statistics.
With raw connectors, the AI…
- Spends budget on schema exploration and data wrangling
- Returns generic metrics without named accounts
- Fills data gaps with fabricated statistics
- Misses insights buried in unstructured conversation data
With Amdahl, the AI…
- Pulls verbatim quotes tied to named speakers and companies
- Surfaces patterns humans wouldn't know to ask about
- Delivers recommendations with specific accounts and next steps
- Cites every claim back to a real conversation
Methodology.
Tested across multiple businesses with different CRM systems (HubSpot, Pipedrive, Salesforce) and call-recording tools (Fathom, Gong). Results were consistent across datasets.
- Same model (Opus) for both paths
- Same question set, same compute budget, same turn limit
- Identical one-line system prompt: "You are an expert business analyst. Answer the user's question using the available tools."
- No coaching, no workflow guidance, no output format prescription
Blinding. Judge model received outputs labeled ‘Path A’ and ‘Path B’ in randomized order. Tool names sanitized (no ‘amdahl’ in traces). Each metric scored independently before seeing the other path’s score.
Fairness check. Standard connectors won 2 of 10 questions (pipeline-churn-over-serviced, pipeline-zombie-accounts). Both were queries where raw CRM data was sufficient. All results are reported.
Benchmark questions.
The full text of all 10 questions used in the benchmark evaluation. Each question was given verbatim to both paths with no additional coaching or formatting guidance.
What do first calls look like for deals that eventually close versus deals we lose? Are there topics, questions, or dynamics in that first conversation that predict whether we'll win or lose?
What topics come up repeatedly in our sales calls that we don't seem to have good answers for? What are buyers asking about that our team struggles to address?
What language do our INTERNAL reps use to describe our product versus the language EXTERNAL buyers use — where's the gap?
Which speaker titles / roles appeared in EVERY Closed Won deal but were missing (or rare) in Closed Lost deals? These are our hidden champions — the roles whose presence signals deal momentum.
How do external speakers describe their CURRENT workflow, tools, or solution before talking to us? I want to understand the 'before state' buyers arrive with.
What words and phrases do EXTERNAL speakers use most often in EARLY-STAGE calls (first 1–2 meetings with a prospect)? What problem framing do buyers arrive with before we've influenced the conversation?
Which competitor names appear most often in our call transcripts, and at what deal stage do they come up?
What do buyers ask about during the EVALUATION stage of a deal that a great case study or one-pager could have pre-answered?
Which companies churned or went dark despite having a HIGH interaction count during the deal cycle? What does 'over-serviced, under-delivered' look like in our data?
Which companies have been in the Prospect / Discovery stage for more than 90 days with NO stage movement? What does our zombie pipeline look like?
Judge scoring rubric.
The AI judge scored each path independently using the framework below. Outputs were labeled ‘Path A’ and ‘Path B’ in randomized order with all tool names sanitized.
Claim-level evaluation
Each individual claim is scored on 5 dimensions (0–3 each, summing to 15). Claims are then classified based on their total score.
| Dimension | Scale |
|---|---|
| Specificity | Vague paraphrase (0) → Named entity + exact number / verbatim quote (3) |
| Non-obviousness | Something a smart human would ask first (0) → Genuinely surprising angle (3) |
| Actionability | 'We should think about X' (0) → 'Send case study Y to 3 specific deals by Friday' (3) |
| Multi-source synthesis | Single table, one query (0) → Joins calls + CRM + meetings + KB in one insight (3) |
| Surprise | Confirms default assumption (0) → Contradicts or sharpens a default assumption (3) |
Claim classification
| Classification | Score threshold | Meaning |
|---|---|---|
| Tablestakes | < 6 / 15 | Basic, expected finding |
| Useful but expected | 6–9 / 15 | Solid analysis, but predictable |
| Interesting | ≥ 10 / 15 | Novel, actionable, non-obvious insight |
Optimization signals
In addition to quality scoring, the judge captures per-path diagnostic signals to identify failure modes and inefficiencies:
- Wasted turns (turns producing no claim)
- High-value tools (tools leading to ≥1 interesting claim)
- Low-value tools (tools called ≥3× producing 0 interesting claims)
- Redundant calls (same or near-duplicate queries)
- Failure modes (hit_max_turns, no_final_brief, duplicate_brief, sub_agent_timeout, empty_query_result_ignored)
- Cost per claim and cost per interesting claim (USD)
- Time to first claim (seconds)
See Amdahl on your own data.
claude plugin marketplace add amdahlco/amdahl-cookbook; claude plugin install amdahl-gtm@amdahl-cookbookOnce Amdahl is connected, see what you can try first