The Demo Index Measurement Engine

Technical Specification v1.0

Technical Specification Document: This document defines the operational architecture for Demo Index measurement systems. It provides scoring engine logic, telemetry mapping specifications, CRM integration requirements, evidence-based validation rules, and benchmark data ingestion models. Intended audience: Engineers, data architects, product managers, and platform developers implementing Demo Index measurement capabilities.

1. System Architecture Overview

The Demo Index Measurement Engine is a data processing system that ingests organizational telemetry, applies scoring logic, and outputs standardized maturity classifications. The architecture consists of five core subsystems:

Core Subsystems

  • Data Ingestion Layer: Telemetry collection from CRM, demo platforms, analytics tools, infrastructure logs
  • Scoring Engine: Calculation logic applying dimension-specific algorithms and weighting models
  • Validation Layer: Evidence-based verification comparing declarative and empirical scores
  • Benchmark Aggregation Engine: Anonymous data pooling and statistical analysis for peer comparison
  • Reporting & API Layer: Output standardization, visualization, and integration endpoints

Design Principles

  • Transparency: All scoring logic is deterministic and auditable
  • Privacy: No PII or customer data leaves organizational boundaries without explicit consent
  • Extensibility: Modular architecture supports future dimension additions and algorithm refinements
  • Hybrid Measurement: Supports both declarative (survey) and empirical (system-based) data sources
  • Degraded Mode Operation: Provides scoring even with incomplete telemetry (with confidence penalties)

Supported Data Sources

Source Category Examples Data Extracted Integration Method
CRM Systems Salesforce, HubSpot, Pipedrive Opportunity timestamps, demo request logs, account context, activity history REST API, Webhooks
Demo Platforms Demostack, Consensus, Saleo, Reprise Environment provisioning logs, session analytics, personalization configs, usage telemetry API, Event Streams
Infrastructure AWS, GCP, Azure Provisioning timestamps, deployment automation logs, resource utilization CloudWatch, Stackdriver, Azure Monitor APIs
Analytics Platforms Segment, Amplitude, Mixpanel Buyer behavior patterns, navigation paths, engagement metrics API, Data Warehouse Sync
Orchestration Tools Zapier, Workato, Custom Workflow automation evidence, integration depth, trigger configurations API, Config Inspection
Survey Responses Assessment Tool Declarative capability scores, process documentation, maturity self-assessment Direct Input

Data Schema Requirements

Ingested data must be normalized to a standardized schema before processing. The following core entities are required:

// Core Entity: DemoRequest { "request_id": "string", "account_id": "string", "opportunity_id": "string", "requested_at": "ISO8601 timestamp", "delivered_at": "ISO8601 timestamp", "time_to_demo_hours": "float", "personalization_level": "enum: [generic, industry, persona, account, realtime]", "automation_percentage": "float 0-100", "self_service_initiated": "boolean", "session_count": "integer", "context_preserved": "boolean" }

2. Scoring Engine Logic

The scoring engine applies dimension-specific algorithms to convert raw telemetry and survey responses into normalized 0-100 scores. Switch between dimensions below, then expand the algorithm only when you need the underlying logic.

Dimension 01

Experience Delivery Scoring

Primary Metric: Time-to-Demo (hours from request to delivery)

Empirical Scoring Algorithm:

Show Algorithm
// Calculate median Time-to-Demo over trailing 90 days median_ttd = MEDIAN(time_to_demo_hours) // Score mapping (inverse relationship: lower time = higher score) IF median_ttd >= 40: score = 10 // DI-1: 1+ week ELSE IF median_ttd >= 16: score = 30 // DI-2: 2-5 days ELSE IF median_ttd >= 4: score = 50 // DI-3: 4-16 hours ELSE IF median_ttd >= 1: score = 75 // DI-4: 1-4 hours ELSE: score = 95 // DI-5: <1 hour

Declarative Fallback: If insufficient telemetry, use survey response: "How quickly can your team deliver a customized demo?" (mapped to same score bands)

3. Validation Layer

Declarative vs. Empirical Score Reconciliation

When both declarative (survey) and empirical (system) scores exist for a dimension, the validation layer applies confidence-weighted blending:

Show Algorithm
// Confidence weights based on data quality empirical_confidence = data_completeness_percentage * data_recency_factor declarative_confidence = MAX(0.6, survey_response_quality_score) // Weighted blend IF empirical_data_available: final_score = ( (empirical_score * empirical_confidence) + (declarative_score * declarative_confidence) ) / (empirical_confidence + declarative_confidence) ELSE: final_score = declarative_score * 0.9 // 10% confidence penalty

Outlier Detection

The validation layer flags significant discrepancies between declarative and empirical scores:

  • Warning Threshold: ±20 point difference between survey and system scores
  • Error Threshold: ±40 point difference (likely measurement error or misrepresentation)
  • Action: Flag for manual review, request additional evidence, or apply confidence penalty

Measurement Integrity Principle: Organizations claiming DI-4 or DI-5 maturity require empirical validation. Declarative-only scores are capped at DI-3 equivalent (60 points) to prevent unsubstantiated claims at advanced maturity levels.

4. Benchmark Aggregation Engine

Data Anonymization & Privacy

Organizations contributing to benchmark data must explicitly consent to anonymized aggregation. All identifying information is stripped before ingestion into the benchmark database:

Anonymization Process

  • Company names, domains, and account identifiers replaced with UUID tokens
  • Individual user data (names, emails) completely removed
  • Geographic data generalized to region level (e.g., "North America" not "San Francisco")
  • Employee count bucketed (e.g., "100-500" not exact headcount)
  • Revenue data bucketed (e.g., "$50M-$100M ARR" not exact figures)

Benchmark Calculation Methodology

Benchmark scores are calculated by vertical, company size, and maturity band. Minimum cohort sizes apply:

Benchmark Type Minimum Cohort Size Statistical Method
Overall Industry Benchmark 100 organizations Median, P25, P75 scores by dimension
Vertical-Specific Benchmark 30 organizations in vertical Median score, distribution by maturity band
Company Size Benchmark 25 organizations in size bracket Median score, range (P10-P90)
Maturity Band Distribution 50 organizations Percentage distribution across DI-1 to DI-5

Current State (2026): Benchmark data does not yet exist. The specification above defines requirements for future benchmark publication once sufficient cohort participation is achieved.

Benchmark Data Schema

// Benchmark Output Format { "benchmark_id": "industry_2026_q2", "cohort_size": 150, "filters": { "vertical": "B2B SaaS", "size_bracket": "100-500 employees", "region": "North America" }, "scores": { "overall_di_score": { "median": 48.5, "p25": 32.0, "p75": 61.0 }, "experience_delivery": { "median": 55.0, "p25": 40.0, "p75": 70.0 }, ... }, "maturity_distribution": { "DI-1": 12, "DI-2": 28, "DI-3": 45, "DI-4": 13, "DI-5": 2 } }

5. Reporting & API Layer

Output Formats

The Measurement Engine provides three standardized output formats:

  • Individual Organization Report: Full dimensional scores, maturity classification, peer comparison (if benchmarks available), improvement recommendations
  • Benchmark Report: Aggregate industry data, maturity distribution, vertical analysis, year-over-year trends
  • API Response: JSON format for programmatic integration with BI tools, dashboards, CRM systems

API Endpoints

// Submit Assessment Data POST /api/v1/assessment Content-Type: application/json Authorization: Bearer {api_key} Body: {organization_data, survey_responses, telemetry_data} Response: {assessment_id, status, estimated_completion_time} // Retrieve Score Report GET /api/v1/assessment/{assessment_id}/report Response: {di_score, dimension_scores, maturity_band, recommendations} // Get Benchmark Data (requires cohort participation consent) GET /api/v1/benchmarks/{vertical}/{size_bracket} Response: {benchmark_scores, peer_comparison, percentile_rank}

6. Implementation Roadmap

Building a production-grade Demo Index Measurement Engine requires phased implementation. The following roadmap balances speed-to-market with measurement integrity:

Phase 1: MVP Measurement System (Months 1-3)

Scope:

  • Build declarative scoring engine (survey-based assessment)
  • Implement core dimensional algorithms (Experience Delivery, Context Intelligence, Orchestration)
  • Create basic validation layer (outlier detection, confidence scoring)
  • Deploy assessment tool with immediate scoring output
  • No benchmarking at this stage (insufficient data)

Output: Functional assessment tool capable of scoring individual organizations

Phase 2: Empirical Telemetry Integration (Months 4-6)

Scope:

  • Build CRM integration connectors (Salesforce, HubSpot)
  • Build demo platform integrations (Demostack, Consensus, etc.)
  • Implement empirical scoring algorithms alongside declarative
  • Deploy confidence-weighted blending logic

Output: Hybrid measurement system with empirical validation

Phase 3: Narrative Integrity Operationalization (Months 7-9)

Scope:

  • Implement Context Persistence Score tracking
  • Deploy Journey Continuity Index measurement
  • Build Cross-Asset Coherence Rate calculation
  • Enable Session State Stability monitoring
  • Implement Stakeholder Context Mapping

Output: Full Narrative Integrity measurement capability

Phase 4: Benchmark Engine & First Report (Months 10-12)

Scope:

  • Deploy anonymization layer and privacy controls
  • Build benchmark aggregation engine
  • Collect 100+ organization assessments for cohort
  • Publish first State of Demo Maturity benchmark report

Output: Industry benchmarks and comparative analytics

Security & Compliance Requirements

  • Data Encryption: All data in transit (TLS 1.3) and at rest (AES-256)
  • Access Control: Role-based access (RBAC) with audit logging
  • Compliance: SOC 2 Type II, GDPR, CCPA adherence
  • Data Retention: Configurable retention policies (default 24 months)

Open Questions for Future Versions

  • Should AI-generated recommendations be part of core engine or separate module?
  • What confidence threshold should trigger manual audit vs auto-scoring?
  • How to handle multi-product organizations with disparate demo maturity levels?
  • Should benchmark participation be incentivized or strictly voluntary?