​

Custom Routers

Custom routers allow you to build intelligent model routing logic trained on your organization’s proprietary data and evaluation results. Beyond our pre-built routers (pulze-v0.1 and pulze-v1.0), you can create routing systems that automatically select the optimal AI model based on your unique requirements.

​

What Are Custom Routers?

Custom routers are intelligent systems that:

• Automatically Select Models: Choose the best AI model for each request
• Learn from Your Data: Train on your proprietary evaluation datasets
• Adapt to Your Needs: Optimize for your specific use cases and quality bars
• Maintain Control: Allow per-space or per-request overrides when needed

​

Pre-Built Routers

​

pulze-v0.1

Our open-sourced router with proven performance:

• Public Benchmarks: Trained on publicly available evaluation data
• Proven Track Record: Battle-tested across thousands of organizations
• Open Source: Fully transparent routing logic
• Default Option: Enabled by default for all organizations

​

pulze-v1.0

Enhanced router with advanced capabilities:

• Automatic Model Discovery: New models are automatically routed without manual targeting
• Improved Evaluation Quality: Significantly enhanced prompt quality during training
• Seamless Integration: Works alongside existing router infrastructure
• Security-First: Not enabled by default - requires testing and approval

​

Building Custom Routers

​

Why Build Custom Routers?

Build custom routers when you need:

• Domain-Specific Optimization: Route based on your industry or use case
• Proprietary Quality Standards: Match your organization’s unique quality bars
• Cost-Performance Trade-offs: Optimize for your specific budget constraints
• Compliance Requirements: Route based on data residency or security policies

​

Router Types

​

How Custom Routers Work

​

1. Create Evaluation Datasets

Build datasets from multiple sources:

• Benchmark Data: Public benchmarks relevant to your domain
• Liked Prompts: Conversations your team has favorited
• Manual Additions: Hand-crafted test cases for your use cases
• Production Data: Real queries from your users

​

2. Run Evaluations

Test models against your datasets:

• Multiple Models: Evaluate several models simultaneously
• Quality Metrics: Track accuracy, relevance, tone, compliance
• Performance Analysis: Compare speed, cost, and quality
• Audit Trails: Maintain complete records of why models pass or fail

​

3. Train Custom Router

Use evaluation results to build routing logic:

• Automatic Training: Router learns from evaluation performance
• Quality Thresholds: Set minimum quality bars for model selection
• Cost Optimization: Balance quality and budget constraints
• Continuous Improvement: Retrain as new models become available

​

4. Deploy and Monitor

Roll out your custom router:

• Gradual Rollout: Test with select spaces before org-wide deployment
• Performance Tracking: Monitor routing decisions and outcomes
• Override Capability: Users can still target specific models when needed
• Iteration: Refine based on real-world performance

​

Dataset Builder

Create comprehensive evaluation datasets tailored to your needs.

​

Data Sources

​

Dataset Management

• Version Control: Track changes to datasets over time
• Tagging System: Organize by category, difficulty, use case
• Sharing Options: Share datasets across your organization
• Import/Export: Bring in external benchmarks or export for analysis

​

Evaluation Engine

Run comprehensive evaluations with full audit trails.

​

Evaluation Features

Quality Metrics

• Response accuracy and correctness
• Tone and style consistency
• Compliance with guidelines
• Citation and source quality
• Format and structure adherence

Performance Metrics

• Response latency
• Token usage and cost
• Throughput and concurrency
• Error rates and reliability

Audit Trail

• Complete evaluation history
• Model comparison reports
• Pass/fail reasoning
• Stakeholder explainability

​

Use Cases

​

Router Configuration

​

Organization Settings

Control default router behavior at the org level: Default Router Selection

• Choose which router (v0.1, v1.0, or custom) your org uses by default
• Set different defaults for different spaces
• Override at the request level when needed

Conversation Naming Model

• Select default model for creating conversation names
• Optimize for speed or quality
• Configure per-space if needed

Compliance-Safe Fallback

• If a model is restricted, system automatically selects from available models
• Stays compliant with org policies
• Maintains user experience without disruption

​

Space-Level Configuration

Customize routing per workspace:

• Space-Specific Routers: Different routing logic per space
• Model Allowlists: Restrict which models can be used
• Quality Thresholds: Set minimum quality requirements
• Cost Controls: Cap spending per space

​

Request-Level Overrides

Maintain flexibility when needed:

• Model Targeting: Explicitly request a specific model
• Router Bypass: Skip routing for specific requests
• A/B Testing: Compare router vs. manual selection
• Debugging: Understand why specific models were chosen

​

Best Practices

​

Start Simple

1. Begin with Pre-Built: Use pulze-v0.1 or pulze-v1.0 first
2. Identify Gaps: Find where pre-built routers don’t meet your needs
3. Build Incrementally: Start with one space or use case
4. Validate Thoroughly: Test extensively before org-wide rollout

​

Build Quality Datasets

• Representative Samples: Include diverse examples from real usage
• Edge Cases: Don’t forget unusual or difficult scenarios
• Regular Updates: Keep datasets current as your product evolves
• Balanced Coverage: Include easy, medium, and hard examples

​

Monitor and Iterate

• Track Routing Decisions: Understand which models are selected and why
• Gather Feedback: Ask users about model performance
• Compare Results: Run periodic evaluations to validate router performance
• Continuous Improvement: Retrain as new models and data become available

​

Maintain Control

• Keep Override Capability: Users should be able to target specific models
• Document Decisions: Explain routing logic to stakeholders
• Set Guardrails: Define clear quality and cost boundaries
• Plan Fallbacks: Handle cases where routing fails or models are unavailable

​

Advanced Features

​

Multi-Objective Optimization

Balance multiple factors:

• Quality vs. cost trade-offs
• Speed vs. accuracy requirements
• Compliance vs. capability needs
• User preference vs. org standards

​

Context-Aware Routing

Route based on:

• Query complexity and type
• User role and permissions
• Space configuration and data
• Time of day and load
• Previous conversation context

​

Ensemble Routing

Combine multiple models:

• Use multiple models for same request
• Compare and validate responses
• Confidence-based selection
• Voting mechanisms for final answer

​

Example Use Cases

​

Financial Services

Requirement: High accuracy, audit trails, cost optimization Solution:

• Custom router trained on financial benchmarks
• Quality threshold: 95%+ accuracy on math/reasoning
• Compliance fallback to approved model list
• Detailed audit logs for every routing decision

​

Healthcare

Requirement: HIPAA compliance, high accuracy, data residency Solution:

• Region-specific router respecting data residency
• Only routes to HIPAA-compliant models
• Quality validation on medical terminology
• Automatic fallback if compliant models unavailable

​

Software Development

Requirement: Code quality, multiple languages, speed Solution:

• Language-specific routing (Python → Model A, JS → Model B)
• Quality benchmarks for code correctness
• Speed optimization for interactive coding
• Cost limits for background batch processing

​

Getting Started

​

Next Steps