Skip to content

Issue Resolution System

This document describes the issue resolution system architecture for FarmCove Delta, which enables tracking and resolving validation issues detected during processing workflows.

Status: The database schema and type definitions are implemented. Service layer implementation is planned for future development.

Table of Contents

  1. Overview
  2. System Architecture
  3. Issue Lifecycle
  4. Issue Types and Categories
  5. Creating Issues
  6. Resolving Issues
  7. Service Layer
  8. Integration with Processing System
  9. Implemented Components
  10. UI Patterns (Planned)
  11. Best Practices
  12. Future Enhancements

Overview

The Issue Resolution System provides a structured way to track, manage, and resolve validation issues that arise during entity processing. Issues can be created by both AI processing steps (when confidence is low) and manual validation steps (when data doesn't meet requirements).

Key Features

  • Flexible Issue Tracking: Track issues for any entity type (transactions, budgets, schedules)
  • Rich Context: Store detailed issue context in JSONB format
  • Suggested Resolutions: System/AI can provide resolution suggestions
  • Resolution Tracking: Complete audit trail of who resolved issues, when, and how
  • Severity Levels: Prioritize issues by severity (critical, high, medium, low)
  • Processing Integration: Seamlessly integrates with the processing system workflow
  • Real-time Updates: Realtime subscriptions for issue status changes

System Architecture

Components

Processing Step (Manual Validation)
    ↓
Issue Detection
    ↓
Create Processing Issue
    ↓
Set Transaction Status to "In Approval"
    ↓
User Reviews Issue in UI
    ↓
User Resolves Issue
    ↓
Update Resolution Data
    ↓
Check All Issues Resolved
    ↓
Continue Approval Process

Data Flow

  1. Issue Creation: Processing steps create issues when validation fails
  2. Status Update: Transaction moves to "In Approval" status
  3. User Notification: User sees issues on "In Approval" transactions with pending issues indicator
  4. Resolution: User provides resolution data
  5. Verification: System verifies all critical issues are resolved
  6. Approval Flow: Transaction continues through the approval process

Issue Lifecycle

Issue States

pending → resolved
        → skipped
        → rejected

State Descriptions

  • pending: Issue detected and awaiting resolution
  • resolved: Issue has been fixed/addressed by user
  • skipped: User chose to skip this issue (non-critical)
  • rejected: Issue marked as invalid/not applicable

State Transitions

// Pending → Resolved
{
  status: 'resolved',
  resolution_data: { /* how it was fixed */ },
  resolution_notes: 'Optional user notes',
  resolved_by_user_id: userId,
  resolved_at: new Date()
}

// Pending → Skipped
{
  status: 'skipped',
  resolution_notes: 'Why it was skipped',
  resolved_by_user_id: userId,
  resolved_at: new Date()
}

// Pending → Rejected
{
  status: 'rejected',
  resolution_notes: 'Why it was rejected',
  resolved_by_user_id: userId,
  resolved_at: new Date()
}

Issue Types and Categories

Common Issue Types

1. Data Quality Issues

total_mismatch

{
  "issue_key": "total_mismatch",
  "issue_title": "Total Mismatch",
  "issue_description": "Transaction total does not match sum of line items",
  "severity": "high",
  "context_data": {
    "transaction_total": 875.06,
    "line_items_sum": 850.0,
    "difference": 25.06
  },
  "suggested_resolution": {
    "type": "adjust_total",
    "suggested_value": 850.0
  }
}

description_unclear

{
  "issue_key": "description_unclear",
  "issue_title": "Description unclear",
  "issue_description": "AI couldn't identify specific items - needs clarification",
  "severity": "medium",
  "context_data": {
    "field": "description",
    "current_value": "Equipment",
    "line_item_id": "uuid"
  },
  "suggested_resolution": {
    "type": "manual_entry",
    "placeholder": "e.g., Camera Gimbal Rental"
  }
}

2. Format Issues

format_invalid

{
  "issue_key": "gst_format_invalid",
  "issue_title": "GST Number: Format Issue",
  "issue_description": "Missing country prefix for Indian GST",
  "severity": "medium",
  "context_data": {
    "field": "tax_number",
    "current_value": "92345679",
    "expected_format": "IN followed by 8 digits",
    "country_code": "IN"
  },
  "suggested_resolution": {
    "type": "use_value",
    "value": "IN92345679"
  }
}

3. Matching Issues

vendor_no_match

{
  "issue_key": "vendor_no_match",
  "issue_title": "Vendor: No exact match found",
  "issue_description": "Similar vendors exist in the system",
  "severity": "high",
  "context_data": {
    "field": "billing_entity_name",
    "current_value": "Desert Storm",
    "similar_vendors": [
      {
        "id": "uuid1",
        "name": "Desert Storm Equipment Rentals",
        "confidence": 0.85
      },
      { "id": "uuid2", "name": "Desert Rentals LLC", "confidence": 0.72 },
      { "id": "uuid3", "name": "Storm Props & Equipment", "confidence": 0.65 }
    ]
  },
  "suggested_resolution": {
    "type": "select_from_options",
    "options": ["uuid1", "uuid2", "uuid3"],
    "allow_create_new": true
  }
}

4. Duplicate Detection

duplicate_found

{
  "issue_key": "duplicate_found",
  "issue_title": "Possible Duplicate",
  "issue_description": "Similar transaction found from 3 days ago",
  "severity": "critical",
  "context_data": {
    "similar_transactions": [
      {
        "id": "uuid",
        "transaction_code": "TRX-2024-0122",
        "date": "2024-01-12",
        "amount": 2145.75,
        "vendor": "Desert Storm",
        "similarity_score": 0.95
      }
    ]
  },
  "suggested_resolution": {
    "type": "confirm_or_link",
    "actions": ["not_duplicate", "link_to_existing"]
  }
}

5. Budget Category Issues

budget_category_needed

{
  "issue_key": "budget_category_needed",
  "issue_title": "Line Item 2: Budget category needed",
  "issue_description": "Equipment Transport - $20.69",
  "severity": "medium",
  "context_data": {
    "line_item_id": "uuid",
    "description": "Equipment Transport",
    "amount": 20.69,
    "line_number": 2
  },
  "suggested_resolution": {
    "type": "select_category",
    "ai_suggested": [
      { "category_id": "uuid1", "name": "Transport", "confidence": 0.88 },
      { "category_id": "uuid2", "name": "Equipment Rental", "confidence": 0.65 }
    ]
  }
}

Creating Issues

Note: The service implementation for creating issues is planned but not yet implemented. Below is the intended usage pattern.

From Manual Validation Steps (Planned)

// In a validation processor (planned implementation)
export async function checkTransactionTotals(
  step: ProcessingJobStepWithRelation
): Promise<boolean> {
  const { processing_job_id } = step;
  const transaction = await getTransactionByJobId(processing_job_id);

  // Calculate line items total
  const lineItemsTotal = transaction.items.reduce(
    (sum, item) => sum + item.total,
    0
  );

  // Check if matches transaction total
  const difference = Math.abs(transaction.total - lineItemsTotal);

  if (difference > 0.01) {
    // Create issue using planned service
    await createIssue({
      processable_type: PROCESSABLE_TYPE.TRANSACTIONS,
      processable_id: transaction.id,
      processing_job_id,
      processing_job_step_id: step.id,
      issue_key: 'total_mismatch',
      issue_title: 'Total Mismatch',
      issue_description: 'Transaction total does not match sum of line items',
      severity: PROCESSING_ISSUE_SEVERITY.HIGH,
      context_data: {
        transaction_total: transaction.total,
        line_items_sum: lineItemsTotal,
        difference: transaction.total - lineItemsTotal,
      },
      suggested_resolution: {
        type: 'adjust_total',
        suggested_value: lineItemsTotal,
      },
    });

    // Transaction will be submitted for approval with pending issues
    // The status change to 'In Approval' happens in finishTransactionProcessing

    return false; // Step completed with issues
  }

  return true; // Step completed successfully
}

From AI Processing Steps (Planned)

// In an AI processor (planned implementation)
export async function processVendorMatching(
  step: ProcessingJobStepWithRelation
): Promise<boolean> {
  const { result_data, processing_job_id } = step;

  if (!result_data) return true;

  const aiResponse = JSON.parse(result_data.message);

  // Check if AI has low confidence
  if (aiResponse.confidence < 0.7) {
    await createIssue({
      processable_type: PROCESSABLE_TYPE.TRANSACTIONS,
      processable_id: getTransactionId(processing_job_id),
      processing_job_id,
      processing_job_step_id: step.id,
      issue_key: 'vendor_no_match',
      issue_title: 'Vendor: No exact match found',
      issue_description: 'Similar vendors exist in the system',
      severity:
        aiResponse.confidence < 0.5
          ? PROCESSING_ISSUE_SEVERITY.CRITICAL
          : PROCESSING_ISSUE_SEVERITY.HIGH,
      context_data: {
        field: 'billing_entity_name',
        current_value: aiResponse.vendor_name,
        similar_vendors: aiResponse.similar_matches,
        ai_confidence: aiResponse.confidence,
      },
      suggested_resolution: {
        type: 'select_from_options',
        options: aiResponse.similar_matches.map((v: any) => v.id),
        allow_create_new: true,
      },
    });

    return false;
  }

  return true;
}

Resolving Issues

Resolution Data Structures

Accept Suggestion

{
  "action": "accepted_suggestion",
  "value": "IN92345679",
  "field": "tax_number"
}

Manual Edit

{
  "action": "manual_edit",
  "field": "description",
  "old_value": "Equipment",
  "new_value": "Camera Gimbal Rental",
  "line_item_id": "uuid"
}

Split Line Item

{
  "action": "split_item",
  "original_amount": 875.06,
  "split_into": [
    {
      "description": "Camera Equipment Rental",
      "amount": 500.0,
      "budget_category_id": "uuid1"
    },
    {
      "description": "Lighting Equipment",
      "amount": 375.06,
      "budget_category_id": "uuid2"
    }
  ]
}

{
  "action": "link_existing",
  "entity_type": "vendor",
  "entity_id": "uuid",
  "field": "billing_entity_id"
}

Create New Entity

{
  "action": "create_new",
  "entity_type": "vendor",
  "entity_data": {
    "name": "Desert Storm Rentals",
    "tax_number": "IN92345679",
    "contact_email": "info@desertstorm.com"
  }
}

Mark as Not Duplicate

{
  "action": "not_duplicate",
  "verified_unique": true,
  "reason": "Different project, different vendor"
}

Service Layer

Processing issue management is implemented as part of the main processing service.

Processing Issue Service (packages/app/src/services/processing/base.ts)

// Create a new issue
export async function createProcessingIssues(
  data: CreateProcessingIssueInput[],
  client?: DatabaseClient
): Promise<ProcessingIssue[] | null>;

// Get all issues for an entity with user relations
export async function getProcessingIssuesByEntity(
  processableType: PROCESSABLE_TYPE,
  processableId: string,
  client?: DatabaseClient
): Promise<ProcessingIssue[] | null>;

// Get count of pending issues
export async function getPendingProcessingIssuesCount(
  processableType: PROCESSABLE_TYPE,
  processableId: string,
  client?: DatabaseClient
): Promise<number>;

// Resolve an issue (supports resolved, requested, ignored statuses)
export async function resolveProcessingIssue(
  input: ResolveProcessingIssueInput,
  userId: string,
  client?: DatabaseClient
): Promise<ProcessingIssue | null>;

// Ignore an issue
export async function ignoreProcessingIssue(
  id: string,
  notes: string | undefined,
  userId: string,
  client?: DatabaseClient
): Promise<ProcessingIssue | null>;

// Request justification for an issue
export async function requestJustificationForIssue(
  id: string,
  notes: string | undefined,
  userId: string,
  client?: DatabaseClient
): Promise<ProcessingIssue | null>;

Client Service (packages/app/src/services/processing/client.ts)

Client-side wrapper functions with ServiceResponse pattern:

// All functions return Promise<ServiceResponse<T>>
export async function getProcessingIssuesByEntity(
  processableType: PROCESSABLE_TYPE,
  processableId: string
): Promise<ServiceResponse<ProcessingIssue[]>>;

export async function getPendingProcessingIssuesCount(
  processableType: PROCESSABLE_TYPE,
  processableId: string
): Promise<ServiceResponse<number>>;

export async function resolveProcessingIssue(
  input: ResolveProcessingIssueInput,
  userId: string
): Promise<ServiceResponse<ProcessingIssue>>;

export async function ignoreProcessingIssue(
  id: string,
  notes: string | undefined,
  userId: string
): Promise<ServiceResponse<ProcessingIssue>>;

export async function requestJustificationForIssue(
  id: string,
  notes: string | undefined,
  userId: string
): Promise<ServiceResponse<ProcessingIssue>>;

export async function createProcessingIssues(
  data: CreateProcessingIssueInput[]
): Promise<ServiceResponse<ProcessingIssue[]>>;

Type Definitions (Implemented)

// From packages/app/src/types/processing.ts
export type CreateProcessingIssueInput = Omit<
  ProcessingIssue,
  'id' | 'created_at' | 'updated_at' | 'resolved_at' | 'resolved_by_user_id'
> & {
  status?: ProcessingIssue['status']; // Optional, defaults to 'pending'
};

export type ResolveProcessingIssueInput = {
  id: string;
  status: 'resolved' | 'skipped' | 'rejected';
  resolution_data?: Record<string, unknown>;
  resolution_notes?: string;
};

Planned Transaction Service Integration

// Get transactions with unresolved issues (In Approval status with pending issues)
export async function getTransactionsWithPendingIssues(
  projectId: string
): Promise<ServiceResponse<Transaction[]>> {
  // Transactions in 'In Approval' status may have pending issues
  // Use getPendingIssuesCount to check for issues on each transaction
  return findMany(DatabaseTable.TRANSACTIONS, {
    project_id: projectId,
    status: 'In Approval',
  });
}

// Update status after all issues resolved
export async function updateTransactionStatusAfterIssueResolution(
  transactionId: string
): Promise<ServiceResponse<void>> {
  const { data: pendingCount } = await getPendingIssuesCount(
    PROCESSABLE_TYPE.TRANSACTIONS,
    transactionId
  );

  if (pendingCount === 0) {
    await updateOne(DatabaseTable.TRANSACTIONS, transactionId, {
      status: 'Processed',
    });
  }
}

Integration with Processing System

Processing Flow with Issues

1. Start Processing Job
   ↓
2. Run AI Steps (OCR, Classification, etc.)
   ↓
3. Run Manual Validation Steps
   ↓
4. Manual Step Detects Issue
   ↓
5. Create Processing Issue
   ↓
6. Continue Other Non-Dependent Steps
   ↓
7. Submit for Approval (with pending issues)
   ↓
8. Pause Processing (wait for resolution)
   ↓
[User resolves issues]
   ↓
9. All Issues Resolved
   ↓
10. Entity Status → "Processed"
   ↓
11. Continue Processing or Complete

Template Configuration

-- Example: Add validation step to transaction template
INSERT INTO process_template_steps (
  process_template_id,
  step_key,
  display_name,
  description,
  type,                          -- 'manual' for validation
  execution_group,
  depends_on_steps,
  processor_method,              -- Method that creates issues
  on_failure_action
) VALUES (
  template_id,
  'check_transaction_totals',
  'Validate Totals',
  'Verify transaction total matches line items sum',
  'manual',
  4,
  ARRAY['transaction_ocr_extraction'],
  'checkTransactionTotals',
  'continue'                     -- Continue even if issues created
);

Implemented Components

Database Schema

The processing_issues table has been created with the following structure:

  • processable_type: Type of entity (transactions, budgets, schedules)
  • processable_id: UUID of the entity with issue
  • processing_job_id: Optional link to processing job
  • processing_job_step_id: Optional link to specific step
  • issue_key: Machine-readable identifier
  • issue_title: Human-readable title
  • issue_description: Detailed description
  • severity: critical, high, medium, low
  • context_data: JSONB flexible issue context
  • suggested_resolution: JSONB with system/AI suggestions
  • status: pending, resolved, skipped, rejected (enum)
  • resolution_data: JSONB with resolution details
  • resolution_notes: Optional user notes
  • resolved_by_user_id: User who resolved
  • resolved_at: Timestamp of resolution

Type Definitions

Located in packages/app/src/types/processing.ts:

  • ProcessingIssue: Base issue type
  • ProcessingIssueWithRelations: Issue with job and step relations
  • CreateProcessingIssueInput: Input for creating issues
  • ResolveProcessingIssueInput: Input for resolving issues

Constants

Located in packages/app/src/constants/processing.ts:

  • PROCESSING_ISSUE_STATUS: pending, resolved, skipped, rejected
  • PROCESSING_ISSUE_SEVERITY: critical, high, medium, low
  • PROCESSABLE_TYPE: transactions, budgets, schedules

Schema Validation

Located in packages/app/src/schemas/processingIssue.schema.ts:

  • processingIssueSchema: Zod schema for validation
  • processingIssueWithRelationsSchema: Schema with relations

UI Patterns (Planned)

Note: UI components for issue resolution are planned but not yet implemented.

In Approval Tab with Pending Issues

Display Structure:

Transaction List (status = "In Approval")
  └─ Transaction Item
      ├─ Basic Info
      ├─ Pending Issues Badge (if has pending issues)
      └─ Click → Opens Transaction Detail with Issues View

Issue Resolution View

Layout:

┌─────────────────────────────────────────────────────────┐
│ Issues to Resolve                     2 of 8 resolved   │
├─────────────────────────────────────────────────────────┤
│                                                         │
│ ┌─────────────────────────────────────┐                │
│ │ Line Item 3: Description unclear    │ [Skip for now] │
│ │ AI couldn't identify specific items │                │
│ │                                     │                │
│ │ [Input: e.g., Camera Gimbal Rental] │                │
│ │                                     │                │
│ │ Amount: $875.06            [Split Item]              │
│ └─────────────────────────────────────┘                │
│                                                         │
│ ┌─────────────────────────────────────┐                │
│ │ GST Number: Format Issue            │                │
│ │ Missing country prefix for Indian GST                │
│ │                                     │                │
│ │ [89234567]  [Use suggestion: IN89234567] ✓          │
│ └─────────────────────────────────────┘                │
│                                                         │
│ ┌─────────────────────────────────────┐                │
│ │ Vendor: No exact match found        │                │
│ │ Similar vendors exist in the system │                │
│ │                                     │                │
│ │ ○ Desert Storm Equipment Rentals  ●                  │
│ │ ○ Desert Rentals LLC                                 │
│ │ ○ Storm Props & Equipment                            │
│ │                                     │                │
│ │ [+ Create "Desert Storm Rentals" as new vendor]     │
│ └─────────────────────────────────────┘                │
│                                                         │
└─────────────────────────────────────────────────────────┘

Real-time Updates

// Subscribe to issue updates
const subscription = supabase
  .channel('processing_issues')
  .on(
    'postgres_changes',
    {
      event: '*',
      schema: 'public',
      table: 'processing_issues',
      filter: `processable_id=eq.${transactionId}`,
    },
    (payload) => {
      // Update UI with new issue or resolution
      handleIssueChange(payload);
    }
  )
  .subscribe();

Best Practices

Issue Creation

  1. Be Specific: Use descriptive issue keys and titles
  2. Provide Context: Include all relevant data in context_data
  3. Suggest Solutions: When possible, provide suggested_resolution
  4. Set Severity: Appropriately categorize issue severity
  5. Link to Source: Always link to processing_job_step_id when available

Resolution Tracking

  1. Complete Resolution Data: Store full details of how issue was resolved
  2. Add Notes: Encourage users to add resolution notes for audit trail
  3. Verify Changes: After resolution, verify the fix was applied correctly
  4. Update Entity Status: Always update entity status after resolution

Performance

  1. Index Usage: Ensure queries use appropriate indexes
  2. Batch Operations: Resolve multiple issues in a transaction when possible
  3. Real-time Efficiency: Use selective subscriptions with filters
  4. Cleanup: Archive or delete old resolved issues periodically

Security

  1. RLS Policies: Ensure users can only see/resolve issues in their projects
  2. Audit Trail: Never delete resolution data, maintain complete history
  3. Validation: Validate resolution_data before applying changes
  4. Authorization: Check user permissions before allowing issue resolution

Future Enhancements

  1. Issue Templates: Predefined issue templates for common scenarios
  2. Bulk Resolution: Resolve multiple similar issues at once
  3. AI-Assisted Resolution: AI suggests resolutions based on past patterns
  4. Issue Analytics: Track issue trends and patterns
  5. Workflow Automation: Auto-resolve certain issue types based on rules
  6. Issue Escalation: Escalate unresolved issues after certain time
  7. Mobile Support: Issue resolution on mobile devices
  8. Collaborative Resolution: Multiple users can work on same issue