Skip to main content

Radish CLI Analytics

Radish CLI includes optional analytics tracking to help improve AI-generated blueprint quality. Analytics collect data about prompts and generated YAML to build a training corpus for improving generation quality.

Privacy​

No PII (Personally Identifiable Information) is collected.

What we track:

  • User prompts/descriptions (intent)
  • Generated types.yml and roles.yml blueprints (results)
  • AI provider used (OpenAI, Anthropic, Codex, etc.)
  • Validation results (did blueprints pass schema validation?)
  • Project name (generic names only, no sensitive data)

What we don't track:

  • User identities
  • API keys or credentials
  • Email addresses or contact information
  • File paths or system information
  • Any personally identifiable data

Purpose​

The goal is to build a high-quality training corpus mapping:

Intent → Results

This helps us understand:

  • Which prompts produce valid blueprints
  • Common patterns in successful generations
  • Where AI models struggle
  • How to improve prompt engineering

Opt-Out​

Analytics are enabled by default but can be easily disabled.

CLI​

Check analytics status:

radish-cli analytics status

Disable analytics:

radish-cli analytics disable

Re-enable analytics:

radish-cli analytics enable

Environment Variables​

For CLI:

export RADISH_ANALYTICS_URL=http://your-analytics-server.com/api/analytics

For Wizard (SvelteKit):

# Disable entirely
export VITE_RADISH_ANALYTICS_ENABLED=false

# Custom server
export VITE_RADISH_ANALYTICS_URL=http://your-analytics-server.com/api/analytics

Configuration File​

Analytics settings are stored in ~/.radish-cli/config.json:

{
  "analyticsEnabled": true,
  "analyticsUrl": "http://localhost:3001/api/analytics"
}

Development & Testing​

Mock Analytics Server​

For development and testing, Radish includes a mock analytics server:

# Start the mock server
node scripts/mock-analytics-server.mjs

The server will log all received analytics events to the console.

Then configure your environment:

# For CLI
export RADISH_ANALYTICS_URL=http://localhost:3001/api/analytics

# For Wizard
export VITE_RADISH_ANALYTICS_URL=http://localhost:3001/api/analytics

Testing Analytics​

  1. Start the mock server:

    node scripts/mock-analytics-server.mjs

  2. Generate a blueprint (wizard):

    cd wizard
    npm run dev
    # Use the wizard to generate blueprints

  3. Or validate existing blueprints (CLI):

    radish-cli validate ./my-project

  4. Check the mock server console for the logged analytics event

Analytics Payload Structure​

{
  "timestamp": "2024-01-15T10:30:00.000Z",
  "version": "1.0",
  "event": "blueprint_generation",
  "data": {
    "prompt": "Create a blog platform with posts, comments, and users",
    "projectName": "myblog",
    "types": "# Full types.yml content here",
    "roles": "# Full roles.yml content here",
    "validationPassed": true,
    "validationErrors": [],
    "provider": "openai",
    "source": "wizard",
    "metadata": {
      "aiModel": "gpt-4o"
    }
  }
}

Implementation Details​

Tracking Points​

Analytics are tracked at two key points:

  1. Wizard Schema Generation (wizard/src/routes/+page.server.ts)

    • Tracks when AI generates new blueprints
    • Includes the full prompt and generated YAML
    • Records AI provider and model used

  2. CLI Validation (src/commands/validate.mjs)

    • Tracks when existing blueprints are validated
    • Captures validation results and errors
    • No prompt available (uses placeholder)

Non-Blocking​

Analytics tracking is non-blocking:

  • Failures never affect the user flow
  • Errors are logged but not shown to users
  • Network timeouts don't impact generation/validation

Code Structure​

src/
  lib/
    analytics.mjs          # CLI analytics library
wizard/
  src/
    lib/
      analytics.ts         # Wizard analytics library
scripts/
  mock-analytics-server.mjs # Development server

Production Deployment​

To deploy your own analytics service:

  1. Create an HTTP endpoint accepting POST requests at /api/analytics
  2. Validate incoming payload structure
  3. Store events in your database
  4. Configure Radish to use your endpoint:
# CLI
export RADISH_ANALYTICS_URL=https://your-domain.com/api/analytics

# Wizard
export VITE_RADISH_ANALYTICS_URL=https://your-domain.com/api/analytics

Example Express Server​

import express from 'express';
const app = express();

app.post('/api/analytics', express.json(), (req, res) => {
  const event = req.body;

  // Validate payload
  if (!event.data || !event.data.prompt || !event.data.types) {
    return res.status(400).json({ error: 'Invalid payload' });
  }

  // Store in database
  await db.analyticsEvents.create(event);

  res.json({ success: true, received: event.timestamp });
});

app.listen(3001);

FAQ​

Q: Why are analytics enabled by default? A: To gather data that helps improve Radish. Since no PII is collected, it's safe and beneficial for the community.

Q: Can I see what data is being sent? A: Yes! Run the mock analytics server and watch the console output to see exactly what's tracked.

Q: Will analytics slow down my workflow? A: No. Analytics are sent asynchronously and never block the generation or validation process.

Q: What if the analytics server is down? A: Analytics failures are silently logged. Your workflow continues unaffected.

Q: Can I contribute to the analytics analysis? A: Yes! The analytics data helps improve Radish for everyone. Contact the maintainers if you'd like to help analyze patterns or improve generation quality.

Support​

For questions or concerns about analytics: