Skip to main content

Repositories

Generated repository classes provide the data access layer for MongoDB operations with automatic population, filtering, and query building.

Table of Contents​

Overview​

Repositories handle direct MongoDB operations:

  • CRUD operations - Create, read, update, delete entities
  • Query building - Automatic filter construction with smart array handling
  • Reference population - Automatic population of related entities
  • Index management - Leverages MongoDB indexes for performance
  • Cursor pagination - Efficient pagination with cursor tokens
  • Archiving - Soft delete functionality

Generated Repositories​

For each entity in your blueprint, Radish generates:

.radish/lib/datalayer/core/repos/
├── _base.repo.ts          # Shared MongoDB operations
├── product.repo.ts         # Product-specific repository
├── user.repo.ts            # User-specific repository
└── ...

Import Paths​

// Using @core alias (internal)
import { ProductRepo } from '@core/repos/product.repo';
import { UserRepo } from '@core/repos/user.repo';

// Using relative path
import { ProductRepo } from '.radish/lib/datalayer/core/repos/product.repo';

Note: Repositories are internal — prefer using the service layer for application code. Services wrap repositories with permission checks, input validation, ownership management, and versioning. Direct repo access is appropriate for scripts, seeding, migrations, and testing.

Repository Architecture​

BaseRepo​

All entity repositories extend BaseRepo<TDoc, TEntity>:

  • TDoc - Mongoose document type (raw MongoDB document)
  • TEntity - Clean TypeScript entity type (transformed for application use)
export abstract class BaseRepo<TDoc extends Document, TEntity> {
  protected abstract model: Model<TDoc>;
  protected abstract toEntity(doc: TDoc): TEntity;

  // Core methods
  async list(criteria, searchFields?): Promise<ListResult<TEntity>>
  async get(id, populate?): Promise<TEntity | null>
  async create(input): Promise<TEntity>
  async update(id, patch): Promise<TEntity | null>
  async remove(id): Promise<{ ok: boolean }>
  
  // Archive methods
  async archive(id): Promise<TEntity | null>
  async restore(id): Promise<TEntity | null>
  async listArchived(criteria): Promise<ListResult<TEntity>>
  
  // Atomic operations
  async increment(id, fields): Promise<TEntity | null>
  
  // Utility methods
  async count(criteria): Promise<number>
  async bulkCreate(inputs, options?): Promise<TEntity[]>
  async bulkUpsert(inputs, options?): Promise<{ inserted: number; updated: number; errors: any[] }>
}

ListResult<T>​

All list methods return a ListResult<T>:

interface ListResult<T> {
  items: T[];
  totalCount: number;
  page: number;
  totalPages: number;
  limit: number;
  truncated: boolean;    // true when requested limit exceeded MAX_PAGE_SIZE (200) and was clamped
  nextCursor?: string;   // cursor token for next page (present when more results exist)
}

Entity Repositories​

Each entity repository includes:

export class ProductRepo extends BaseRepo<any, Product> {
  protected model = ProductModel;
  protected toEntity = toProduct;

  // Override list to provide entity-specific search fields
  async list(criteria: any = {}) {
    const searchFields = ['title', 'description', 'sku'];
    return super.list(criteria, searchFields);
  }
}

Document Transformation​

Repositories automatically transform MongoDB documents to clean entities:

// MongoDB document (internal)
{
  _id: ObjectId('507f...'),
  title: 'Product',
  ownerId: ObjectId('608a...'),
  createdAt: ISODate('2024-01-15T10:30:00Z'),
  __v: 0
}

// Transformed entity (for application use)
{
  id: '507f...',           // _id → id
  title: 'Product',
  ownerId: '608a...',      // ObjectId → string
  createdAt: '2024-01-15T10:30:00Z'  // Date → ISO string
  // __v removed
}

CRUD Operations​

get(id, populate?)​

Inverse relations are populatable the same way. Course.lessons resolves like Lesson.courseId does — a virtual path behaves exactly as a real ref at the call site. Inverse accessors are generated automatically from every ref, with nothing to declare. See Relations.

Get a single entity by ID with optional reference population.

get(id: string, populate?: string | string[] | { paths: string[]; depth?: number }): Promise<Product | null>

Example:

import { ProductRepo } from '@core/repos/product.repo';

const repo = new ProductRepo();

// Get by ID
const product = await repo.get('507f1f77bcf86cd799439011');

// Get with populated reference
const productWithAuthor = await repo.get('507f...', 'authorId');

// Get with multiple populated references
const productWithRefs = await repo.get('507f...', ['authorId', 'categoryId']);

create(input)​

Create a new entity.

create(input: any): Promise<Product>

Example:

const product = await repo.create({
  title: 'New Product',
  price: 29.99,
  ownerId: 'user-123',
  status: 'DRAFT'
});

console.log(product.id);  // Auto-generated MongoDB ObjectId

update(id, patch)​

Update an existing entity with partial data.

update(id: string, patch: any): Promise<Product | null>

Example:

const updated = await repo.update('507f...', {
  price: 39.99,
  status: 'ACTIVE'
});

if (!updated) {
  console.log('Product not found');
}

Note: Uses MongoDB $set operator internally for partial updates.

remove(id)​

Permanently delete an entity (hard delete).

remove(id: string): Promise<{ ok: boolean }>

Example:

const result = await repo.remove('507f...');
console.log(result.ok); // true

Warning: Hard deletes are permanent. Consider using archive() instead for soft deletes.

increment(id, fields)​

Atomically increment or decrement numeric fields using MongoDB $inc:

// Increment a single field
await repo.increment(id, { viewCount: 1 });

// Decrement
await repo.increment(id, { stock: -5 });

// Multiple fields at once
await repo.increment(id, { viewCount: 1, downloadCount: 1, stock: -1 });

This is atomic at the database level — no read-modify-write race conditions.

List & Filtering​

list(criteria, searchFields?)​

List entities with filtering, pagination, sorting, and search.

list(criteria: ListCriteria, searchFields?: string[]): Promise<ListResult<Product>>

ListCriteria interface:

interface ListCriteria {
  limit?: number;                    // Results per page (1-100, default: 20)
  cursor?: string;                   // Pagination cursor token
  skip?: number;                     // Offset-based pagination
  offset?: number;                   // Alias for skip
  q?: string;                        // Text search query
  ownerId?: string;                  // Filter by owner
  includeArchived?: boolean;         // Include archived items (default: false)
  populate?: string | string[] | PopulateConfig;  // Reference population
  sort?: {
    field: string;
    direction: 'asc' | 'desc';
    collation?: CollationOptions;    // MongoDB collation for text sorting
  };
  dateFrom?: string;                 // ISO date string — records from this date
  dateTo?: string;                   // ISO date string — records up to this date
  dateMonth?: string;                // Month shorthand (e.g., '2026-03')
  dateField?: string;                // Target field for date filters (default: 'createdAt')
  [key: string]: any;                // Additional field filters
}

Examples:

import { ProductRepo } from '@core/repos/product.repo';

const repo = new ProductRepo();

// Basic list
const result = await repo.list({ limit: 25 });
console.log(result.items);      // Product[]
console.log(result.nextCursor); // Cursor for next page

// Filter by field
const activeProducts = await repo.list({
  status: 'ACTIVE',
  limit: 50
});

// Filter by multiple fields
const filtered = await repo.list({
  status: 'ACTIVE',
  categoryId: 'cat-123',
  ownerId: 'user-456'
});

// Text search (searches title, description, sku)
const searchResults = await repo.list({
  q: 'laptop',
  limit: 10
});

// Pagination with cursor
const page1 = await repo.list({ limit: 20 });
const page2 = await repo.list({
  limit: 20,
  cursor: page1.nextCursor
});

Smart Array Filtering​

Repositories automatically detect array fields and apply $in operator:

// Product blueprint has tags: string[]

// Filter products with ANY of these tags (uses MongoDB $in)
const tagged = await repo.list({
  tags: ['featured', 'new-arrival']
});
// MongoDB query: { tags: { $in: ['featured', 'new-arrival'] } }

// Scalar field filter (exact match)
const byCategory = await repo.list({
  categoryId: 'cat-123'
});
// MongoDB query: { categoryId: 'cat-123' }

Sorting​

// Sort by field (ascending)
const sorted = await repo.list({
  sort: {
    field: 'price',
    direction: 'asc'
  }
});

// Sort with collation (case-insensitive, natural number ordering)
const naturalSort = await repo.list({
  sort: {
    field: 'title',
    direction: 'asc',
    collation: {
      locale: 'en',
      strength: 2,              // Case-insensitive
      numericOrdering: true     // Natural number sort (1, 2, 10 not 1, 10, 2)
    }
  }
});

String shorthand:

// Prefix with - for descending
const items = await repo.list({ sort: '-createdAt' });
// Equivalent to: { sort: { field: 'createdAt', direction: 'desc' } }

Sorting Notes:

  • Default sort is by _id (insertion order)
  • Sorted fields must exist in the schema
  • Secondary sort by _id for consistent ordering when primary field has duplicates
  • When using custom sort with cursor, cursor is ignored (starts from first page)

Reference Population​

Repositories support automatic population of referenced entities to avoid additional queries.

Basic Population​

// Single reference
const product = await repo.get('507f...', 'authorId');

console.log(product.authorId);
// Instead of: '608a...' (ObjectId string)
// Returns: { id: '608a...', name: 'John Doe', email: 'john@example.com', ... }

Multiple References​

// Multiple references (comma-separated or array)
const product = await repo.get('507f...', 'authorId,categoryId');

// Or as array
const product = await repo.get('507f...', ['authorId', 'categoryId']);

Nested Population​

// Populate author and author's department (dot notation)
const product = await repo.get('507f...', 'authorId.departmentId');

console.log(product.authorId.departmentId);
// { id: '807f...', name: 'Engineering', ... }

Population in List Queries​

// List with populated references
const result = await repo.list({
  status: 'ACTIVE',
  populate: 'authorId,categoryId',
  limit: 25
});

result.items.forEach(product => {
  console.log(product.authorId.name);      // Populated
  console.log(product.categoryId.name);   // Populated
});

Advanced Population Configuration​

// Population with depth control
const product = await repo.get('507f...', {
  paths: ['authorId.departmentId.managerId', 'categoryId'],
  depth: 3  // Maximum nesting depth
});

Population Notes:

  • Non-existent references return null
  • Array references populate each item in the array
  • Populated fields respect the same transformation rules (ObjectId → string, etc.)
  • Performance impact: Each population level adds a database query

Archive & Restore​

Repositories support soft deletion through archiving.

archive(id)​

Soft delete an entity by setting archivedAt timestamp.

const archived = await repo.archive('507f...');

console.log(archived.archivedAt); // '2024-01-15T10:30:00Z'

restore(id)​

Restore an archived entity by clearing archivedAt.

const restored = await repo.restore('507f...');

console.log(restored.archivedAt); // null

listArchived(criteria)​

List only archived entities.

// List all archived products
const archived = await repo.listArchived({ limit: 50 });

// Filter archived products
const archivedByCategory = await repo.listArchived({
  categoryId: 'cat-123'
});

Default Sorting: Archived items default to sorting by archivedAt descending (most recently archived first).

listIncludingArchived(criteria)​

List both active and archived entities.

const all = await repo.listIncludingArchived({ limit: 100 });

Note: Standard list() excludes archived items by default. Use includeArchived: true or listIncludingArchived() to include them.

Bulk Operations​

bulkCreate(inputs, options?)​

Fast bulk insertion for migrations, seeding, and disaster recovery.

bulkCreate(
  inputs: any[],
  options?: {
    preserveIds?: boolean;  // Keep existing _id fields (default: false)
    ordered?: boolean;      // Stop on first error (default: false)
  }
): Promise<Product[]>

Example:

// Fast seeding with auto-generated IDs
const products = await repo.bulkCreate([
  { title: 'Product 1', price: 19.99 },
  { title: 'Product 2', price: 29.99 },
  { title: 'Product 3', price: 39.99 }
]);

console.log(`Inserted ${products.length} products`);

Disaster Recovery (preserve IDs):

// Restore from backup with original IDs
const restored = await repo.bulkCreate(backupData, {
  preserveIds: true  // Keep original _id values
});

Performance:

  • Uses MongoDB insertMany (100-250x faster than individual creates)
  • Bypasses service layer validation and versioning
  • Best for trusted data sources (migrations, backups, seeding)

Error Handling:

// Stop on first error (ordered insertion)
await repo.bulkCreate(data, { ordered: true });

// Continue on errors (unordered insertion, default)
const results = await repo.bulkCreate(data, { ordered: false });
// Some records may fail, others succeed

bulkUpsert(inputs, options?)​

Update existing records or insert new ones, matched by a field key. Safer than bulkCreate for production restores since it won't fail on duplicate keys.

bulkUpsert(
  inputs: any[],
  options?: {
    matchField?: string;      // Single field to match on (default: 'id')
    matchFields?: string[];   // Compound key matching (overrides matchField)
    preserveIds?: boolean;    // Keep existing _id fields (default: false)
    ordered?: boolean;        // Stop on first error (default: false)
  }
): Promise<{ inserted: number; updated: number; errors: any[] }>

Example:

// Sync data by unique key
const result = await repo.bulkUpsert(importData, { matchField: 'email' });
console.log(`Inserted: ${result.inserted}, Updated: ${result.updated}`);

// Restore from backup (match by ID, preserve original IDs)
const result = await repo.bulkUpsert(backupData, {
  matchField: 'id',
  preserveIds: true
});

// Compound-key upsert (match by multiple fields)
const result = await repo.bulkUpsert(salesData, {
  matchFields: ['productId', 'regionId', 'quarter']
});

How it works:

  • Uses MongoDB bulkWrite with updateOne + upsert: true for each record
  • Existing records matching the key are updated via $set
  • Non-matching records are inserted as new documents
  • When preserveIds is true and matching by id/_id, the original _id is preserved on insert via $setOnInsert

Advanced Queries​

count(criteria)​

Get count of entities matching criteria.

// Count all products
const total = await repo.count({});

// Count with filter
const activeCount = await repo.count({ status: 'ACTIVE' });

// Count by owner
const userProducts = await repo.count({ ownerId: 'user-123' });

Note: Excludes archived items by default (same as list()).

Custom Query Building​

Repositories use buildFilterQuery() internally to construct MongoDB queries:

// Input criteria
{
  status: 'ACTIVE',
  tags: ['featured', 'new'],
  ownerId: 'user-123',
  q: 'laptop'
}

// Generated MongoDB query
{
  status: 'ACTIVE',
  tags: { $in: ['featured', 'new'] },  // Smart array handling
  ownerId: 'user-123',
  $or: [                                // Text search
    { title: /laptop/i },
    { description: /laptop/i },
    { sku: /laptop/i }
  ],
  archivedAt: null                      // Exclude archived
}

Field Type Detection​

Repositories automatically detect field types from Mongoose schema:

// Detected from schema
fieldDefinitions = {
  title: { type: 'string' },
  price: { type: 'number' },
  tags: { type: 'string[]' },           // Array field
  categoryId: { type: 'objectId' },
  createdAt: { type: 'isoDate' }
}

// Used for smart array filtering
if (fieldType.endsWith('[]') && Array.isArray(value)) {
  query[field] = { $in: value };  // Use $in for array fields
}

Using Repositories​

Services should use repositories for data access:

export class ProductService extends BaseService {
  private repo = new ProductRepo();

  async list(input: unknown) {
    // Validate input
    const criteria = productCriteriaSchema.parse(input);

    // Check permissions
    const hasViewAll = await this.hasPermission('product:view:all');
    if (!hasViewAll) {
      criteria.ownerId = this.ownerIdOrThrow();
    }

    // Use repo for data access
    return this.repo.list(criteria);
  }
}

Direct Usage (Testing, Scripts)​

Repositories can be used directly for testing and scripts:

import { ProductRepo } from '@core/repos/product.repo';

const repo = new ProductRepo();

// Seeding script
const products = await repo.bulkCreate([
  { title: 'Product 1', price: 19.99, ownerId: 'system' },
  { title: 'Product 2', price: 29.99, ownerId: 'system' }
]);

console.log(`Seeded ${products.length} products`);

With Transactions​

Use MongoDB transactions for multi-step operations:

import { getConnection } from '@generated/datalayer/core/db';

const session = await getConnection().startSession();
session.startTransaction();

try {
  // Create order
  const order = await orderRepo.create({
    productId: 'prod-123',
    quantity: 2,
    ownerId: 'user-456'
  });

  // Update inventory (manual MongoDB operation)
  await ProductModel.findByIdAndUpdate(
    'prod-123',
    { $inc: { inventory: -2 } },
    { session }  // Pass session for transaction
  );

  await session.commitTransaction();
} catch (error) {
  await session.abortTransaction();
  throw error;
} finally {
  session.endSession();
}

Note: BaseRepo methods don't accept session parameters by default. For transactional operations, use Mongoose models directly with session parameter.

Best Practices​

1. Use Services, Not Repos Directly​

❌ Don't:

// In HTTP route - bypasses permission checks!
const products = await productRepo.list({});

✅ Do:

// In HTTP route - use service for permissions
const service = new ProductService(ctx);
const products = await service.list({});

2. Leverage Smart Array Filtering​

❌ Don't:

// Manual $in operator
const products = await repo.list({
  tags: { $in: ['featured', 'new'] }
});

✅ Do:

// Automatic $in detection
const products = await repo.list({
  tags: ['featured', 'new']  // Automatically uses $in
});

3. Use Population Wisely​

❌ Don't:

// Over-populating (performance hit)
const product = await repo.get(id, 'authorId.departmentId.managerId.teamId.companyId');

✅ Do:

// Populate only what you need
const product = await repo.get(id, 'authorId');

// Or populate multiple shallow references
const product = await repo.get(id, ['authorId', 'categoryId', 'brandId']);

4. Use Bulk Operations for Performance​

❌ Don't:

// Slow - individual creates
for (const item of data) {
  await repo.create(item);  // N database calls
}

✅ Do:

// Fast - bulk create
await repo.bulkCreate(data);  // 1 database call

5. Understand Archiving vs Deletion​

Use archive() when:

  • Data may need to be restored
  • Audit trail is important
  • Related entities reference this entity

Use remove() when:

  • Data is truly disposable
  • Storage is a concern
  • GDPR/data deletion requirements

6. Handle Pagination Correctly​

Both cursor-based and skip/offset pagination are supported.

Skip/offset works well for admin panels and moderate datasets:

const page1 = await repo.list({ limit: 20, skip: 0 });
const page2 = await repo.list({ limit: 20, skip: 20 });

Cursor pagination is better for large datasets and infinite scroll:

const page1 = await repo.list({ limit: 20 });
const page2 = await repo.list({ limit: 20, cursor: page1.nextCursor });

Choose based on your use case — cursor is more efficient at scale, but skip/offset is simpler when you need direct page access.

Next Steps​

  • Services - Business logic and permission enforcement
  • Models - Mongoose schemas and MongoDB structure
  • Contracts - Zod schemas and TypeScript types