Data Flow - ThriveSend B2B2G

Version: 0.1.4 Last Updated: November 20, 2025 Status: Production

Overview

This document illustrates how data flows through the ThriveSend B2B2G platform, from user authentication to campaign execution, content management, and POPIA compliance logging.

1. Service Provider Onboarding Flow

sequenceDiagram
    participant User as New User
    participant Clerk as Clerk Auth
    participant MW as Middleware
    participant API as API Route
    participant DB as PostgreSQL
    participant Audit as Audit Logger

    User->>Clerk: Sign Up
    Clerk->>Clerk: Create Clerk Account
    Clerk-->>User: Redirect to Onboarding

    User->>API: POST /api/service-provider/organization
    API->>Clerk: Verify JWT Token
    Clerk-->>API: Token Valid

    API->>DB: Create Organization
    DB-->>API: Organization Created

    API->>DB: Create OrganizationUser
    DB-->>API: User Created

    API->>DB: Create Default Settings
    DB-->>API: Settings Created

    API->>Audit: Log Organization Creation
    Audit->>DB: Store Audit Log

    API-->>User: Return Organization Data
    User->>User: Redirect to Dashboard

    Note over User,Audit: Complete B2B2G Setup<br/>Organization Ready

2. Client Creation Flow (B2B2G)

flowchart TB
    Start([Service Provider<br/>Creates Client])

    ValidateInput[Validate Input<br/>- Name, Sector, Type<br/>- Security Clearance<br/>- Contact Info]

    CheckSector{Sector Type?}

    GovWorkflow[Government Workflow<br/>- Require Security Clearance<br/>- Enhanced Compliance<br/>- Government Templates]

    BizWorkflow[Business Workflow<br/>- Standard Compliance<br/>- Commercial Templates<br/>- Performance Focus]

    CreateDB[(Create Client Record<br/>in PostgreSQL)]

    CreateAccess[Create ClientAccess<br/>for Service Provider Admin]

    InitAnalytics[Initialize Analytics<br/>Dashboard for Client]

    AuditLog[Log Client Creation<br/>POPIA Audit Trail]

    Notify[Send Notification<br/>to Account Manager]

    Complete([Client Created<br/>Ready for Campaigns])

    Start --> ValidateInput
    ValidateInput --> CheckSector

    CheckSector -->|GOVERNMENT| GovWorkflow
    CheckSector -->|BUSINESS| BizWorkflow

    GovWorkflow --> CreateDB
    BizWorkflow --> CreateDB

    CreateDB --> CreateAccess
    CreateAccess --> InitAnalytics
    InitAnalytics --> AuditLog
    AuditLog --> Notify
    Notify --> Complete

    style GovWorkflow fill:#ffebee
    style BizWorkflow fill:#e8f5e9
    style AuditLog fill:#e1f5fe

3. Campaign Creation & Execution Flow

sequenceDiagram
    participant User as Account Manager
    participant UI as Campaign UI
    participant API as Campaign API
    participant CmpSvc as Campaign Service
    participant DB as Database
    participant Queue as Job Queue
    participant Email as Email Service
    participant Analytics as Analytics Service
    participant Audit as Audit Logger

    User->>UI: Create New Campaign
    UI->>API: POST /api/campaigns

    API->>CmpSvc: Validate Campaign Data
    CmpSvc->>DB: Check Client Exists
    DB-->>CmpSvc: Client Valid

    CmpSvc->>DB: Check Security Clearance
    DB-->>CmpSvc: Clearance Sufficient

    CmpSvc->>DB: Create Campaign Record
    DB-->>CmpSvc: Campaign Created

    CmpSvc->>Audit: Log Campaign Creation
    Audit->>DB: Store Audit Log

    CmpSvc-->>API: Return Campaign
    API-->>UI: Display Campaign

    User->>UI: Add Content & Schedule
    UI->>API: PUT /api/campaigns/{id}
    API->>CmpSvc: Update Campaign
    CmpSvc->>DB: Update Campaign Status

    Note over User,Audit: Campaign Scheduled

    Queue->>Queue: Campaign Start Time
    Queue->>CmpSvc: Execute Campaign

    CmpSvc->>DB: Get Campaign Content
    DB-->>CmpSvc: Content List

    loop For Each Content
        CmpSvc->>Email: Send Email/SMS
        Email-->>CmpSvc: Delivery Status
        CmpSvc->>Analytics: Record Metrics
        Analytics->>DB: Store Analytics
        CmpSvc->>Audit: Log Content Delivery
        Audit->>DB: Store Audit Log
    end

    CmpSvc->>DB: Update Campaign Status
    DB-->>CmpSvc: Status Updated

    CmpSvc->>Analytics: Generate Report
    Analytics->>DB: Store Report

    Note over Queue,Audit: Campaign Complete<br/>Analytics Available

4. Content Approval Flow (Government)

stateDiagram-v2
    [*] --> Draft: Content Creator<br/>Creates Content

    Draft --> PendingApproval: Submit for<br/>Approval

    PendingApproval --> ReviewLevel1: Assigned to<br/>Compliance Officer

    ReviewLevel1 --> Rejected: Compliance<br/>Issues Found
    ReviewLevel1 --> ReviewLevel2: Level 1<br/>Approved

    Rejected --> Draft: Revisions<br/>Required

    ReviewLevel2 --> Rejected: Government<br/>Standards Not Met
    ReviewLevel2 --> Approved: Level 2<br/>Approved

    Approved --> Published: Manual<br/>Publish
    Approved --> Scheduled: Schedule<br/>Publish

    Scheduled --> Published: Scheduled<br/>Time Reached

    Published --> Archived: Campaign<br/>Completed

    Archived --> [*]

    note right of PendingApproval
        POPIA Audit Log:
        - Submission timestamp
        - Creator ID
        - Content version
    end note

    note right of ReviewLevel1
        Government Compliance:
        - Security clearance check
        - Content classification
        - Template adherence
    end note

    note right of Approved
        Final Checks:
        - Legal compliance
        - Brand guidelines
        - POPIA consent
    end note

5. User Authentication & Authorization Flow

flowchart TB
    Start([User Requests<br/>Protected Resource])

    CheckAuth{Has Valid<br/>Session?}

    RedirectLogin[Redirect to<br/>Clerk Sign In]

    ClerkAuth[Clerk JWT<br/>Verification]

    ValidToken{Valid<br/>Token?}

    Unauthorized[Return 401<br/>Unauthorized]

    LoadOrg[Load Organization<br/>from Clerk]

    CheckOrg{Organization<br/>Exists?}

    OnboardingRedirect[Redirect to<br/>Onboarding]

    LoadUser[Load OrganizationUser<br/>from Database]

    CheckUser{User<br/>Exists?}

    CreateUser[Create OrganizationUser<br/>Record]

    LoadPermissions[Load User<br/>Permissions & Clearance]

    CheckPermissions{Has Required<br/>Permission?}

    Forbidden[Return 403<br/>Forbidden]

    AuditAccess[Log Access<br/>POPIA Audit Trail]

    GrantAccess[Grant Access<br/>to Resource]

    Complete([Return Resource<br/>to User])

    Start --> CheckAuth

    CheckAuth -->|No| RedirectLogin
    CheckAuth -->|Yes| ClerkAuth

    RedirectLogin --> ClerkAuth

    ClerkAuth --> ValidToken

    ValidToken -->|No| Unauthorized
    ValidToken -->|Yes| LoadOrg

    LoadOrg --> CheckOrg

    CheckOrg -->|No| OnboardingRedirect
    CheckOrg -->|Yes| LoadUser

    LoadUser --> CheckUser

    CheckUser -->|No| CreateUser
    CheckUser -->|Yes| LoadPermissions

    CreateUser --> LoadPermissions

    LoadPermissions --> CheckPermissions

    CheckPermissions -->|No| Forbidden
    CheckPermissions -->|Yes| AuditAccess

    AuditAccess --> GrantAccess
    GrantAccess --> Complete

    style Unauthorized fill:#ffebee
    style Forbidden fill:#ffebee
    style AuditAccess fill:#e1f5fe
    style GrantAccess fill:#e8f5e9

6. POPIA Data Processing Flow

sequenceDiagram
    participant User as User/System
    participant API as API Layer
    participant Validator as Input Validator
    participant Service as Service Layer
    participant DB as PostgreSQL
    participant Audit as Audit Logger
    participant Consent as Consent Manager
    participant Classification as Data Classifier

    User->>API: Request Data Operation

    API->>Validator: Validate Input
    Validator->>Validator: Check Data Format
    Validator->>Validator: Sanitize Input
    Validator-->>API: Input Valid

    API->>Service: Process Request

    Service->>Consent: Check Consent Required
    Consent->>DB: Query ConsentRecord
    DB-->>Consent: Consent Status

    alt Consent Required but Missing
        Consent-->>Service: Consent Missing
        Service-->>API: Return 403
        API-->>User: Request Consent
    else Consent Valid or Not Required
        Consent-->>Service: Proceed

        Service->>Classification: Classify Data
        Classification->>Classification: Determine Level<br/>(PUBLIC/CONFIDENTIAL/RESTRICTED)
        Classification-->>Service: Classification Result

        Service->>DB: Execute Operation
        DB-->>Service: Operation Result

        Service->>Audit: Log Operation

        Note over Audit: Audit Log Includes:<br/>- User ID<br/>- Action Type<br/>- Resource<br/>- Data Classification<br/>- Timestamp<br/>- IP Address<br/>- Result

        Audit->>DB: Store Audit Log
        DB-->>Audit: Log Stored

        alt Personal Data Processed
            Service->>DB: Create DataProcessingRecord
            DB-->>Service: Record Created
        end

        Service-->>API: Return Result
        API-->>User: Success Response
    end

    Note over User,Classification: Every Data Operation<br/>POPIA Compliant

7. Analytics Data Collection Flow

flowchart LR
    subgraph Sources["Data Sources"]
        Email[Email Delivery<br/>Events]
        SMS[SMS Delivery<br/>Events]
        Web[Web Activity<br/>Tracking]
        API_Events[API Usage<br/>Events]
    end

    subgraph Collection["Event Collection"]
        EmailTracker[Email Tracker]
        SMSTracker[SMS Tracker]
        WebTracker[Web Tracker]
        APITracker[API Tracker]
    end

    subgraph Processing["Data Processing"]
        Aggregator[Event Aggregator]
        Calculator[Metrics Calculator]
        Enricher[Data Enricher]
    end

    subgraph Storage["Data Storage"]
        TimeSeries[(Time Series<br/>Analytics DB)]
        Aggregated[(Aggregated<br/>Metrics DB)]
        Cache[(Redis<br/>Cache)]
    end

    subgraph Presentation["Data Presentation"]
        Dashboard[Dashboard UI]
        Reports[Report Generator]
        Export[Data Export]
    end

    Email --> EmailTracker
    SMS --> SMSTracker
    Web --> WebTracker
    API_Events --> APITracker

    EmailTracker --> Aggregator
    SMSTracker --> Aggregator
    WebTracker --> Aggregator
    APITracker --> Aggregator

    Aggregator --> Calculator
    Calculator --> Enricher

    Enricher --> TimeSeries
    Enricher --> Aggregated
    Enricher --> Cache

    TimeSeries --> Dashboard
    Aggregated --> Dashboard
    Cache --> Dashboard

    TimeSeries --> Reports
    Aggregated --> Reports

    Reports --> Export

    style Aggregator fill:#e1f5fe
    style Cache fill:#fff3e0
    style Dashboard fill:#e8f5e9

8. Team Invitation Flow

sequenceDiagram
    participant Admin as Service Provider Admin
    participant UI as Team UI
    participant API as Invitation API
    participant InvSvc as Invitation Service
    participant DB as Database
    participant Email as Email Service
    participant Clerk as Clerk Auth
    participant Audit as Audit Logger

    Admin->>UI: Invite Team Member
    UI->>API: POST /api/invitations

    API->>InvSvc: Create Invitation
    InvSvc->>InvSvc: Generate Secure Token
    InvSvc->>InvSvc: Set Expiry (7 days)

    InvSvc->>DB: Store Invitation
    DB-->>InvSvc: Invitation Created

    InvSvc->>Audit: Log Invitation Creation
    Audit->>DB: Store Audit Log

    InvSvc->>Email: Send Invitation Email
    Note over Email: POPIA-Compliant<br/>Email Template<br/>SA Data Residency
    Email-->>InvSvc: Email Sent

    InvSvc->>DB: Update EmailTracking
    DB-->>InvSvc: Tracking Updated

    InvSvc-->>API: Return Invitation
    API-->>UI: Display Success

    Note over Admin,Audit: Invitation Sent<br/>Awaiting Acceptance

    participant Invitee as Invited User

    Invitee->>Invitee: Receive Email
    Invitee->>UI: Click Invitation Link

    UI->>API: GET /api/invitations/accept?token=...
    API->>InvSvc: Validate Token

    InvSvc->>DB: Query Invitation
    DB-->>InvSvc: Invitation Data

    InvSvc->>InvSvc: Check Expiry
    InvSvc->>InvSvc: Check Status

    alt Token Valid
        InvSvc-->>API: Token Valid
        API-->>UI: Redirect to Sign Up

        Invitee->>Clerk: Complete Sign Up
        Clerk->>Clerk: Create Account
        Clerk-->>Invitee: Account Created

        Invitee->>API: POST /api/invitations/accept
        API->>InvSvc: Accept Invitation

        InvSvc->>DB: Create OrganizationUser
        DB-->>InvSvc: User Created

        InvSvc->>DB: Update Invitation Status
        DB-->>InvSvc: Status Updated

        InvSvc->>Audit: Log Invitation Acceptance
        Audit->>DB: Store Audit Log

        InvSvc-->>API: User Onboarded
        API-->>UI: Redirect to Dashboard

    else Token Invalid/Expired
        InvSvc-->>API: Token Invalid
        API-->>UI: Display Error
    end

    Note over Invitee,Audit: Team Member<br/>Onboarded Successfully

9. Data Subject Request Flow (POPIA)

flowchart TB
    Start([Data Subject<br/>Submits Request])

    ReceiveRequest[Receive Request<br/>- Access<br/>- Correction<br/>- Deletion<br/>- Portability]

    ValidateIdentity[Validate Identity<br/>- Email verification<br/>- Additional auth if needed]

    CreateRecord[Create DataSubjectRequest<br/>Record in Database]

    AuditLog1[Log Request<br/>POPIA Audit Trail]

    NotifyCompliance[Notify<br/>Compliance Officer]

    ReviewRequest{Request Type?}

    AccessRequest[Access Request<br/>Gather all personal data]

    CorrectionRequest[Correction Request<br/>Update specified data]

    DeletionRequest[Deletion Request<br/>Anonymize/delete data]

    PortabilityRequest[Portability Request<br/>Export data in format]

    ValidateAction[Validate Action<br/>- Legal compliance<br/>- Retention policies<br/>- Dependencies]

    CheckRetention{Within Retention<br/>Period?}

    CannotDelete[Cannot Delete<br/>Legal retention required]

    ExecuteAction[Execute Action<br/>Update/Delete/Export Data]

    AuditLog2[Log Action<br/>Complete Audit Trail]

    NotifyUser[Notify Data Subject<br/>Request Completed]

    UpdateRequest[Update Request Status<br/>to COMPLETED]

    Complete([Request Fulfilled<br/>POPIA Compliant])

    Start --> ReceiveRequest
    ReceiveRequest --> ValidateIdentity
    ValidateIdentity --> CreateRecord
    CreateRecord --> AuditLog1
    AuditLog1 --> NotifyCompliance
    NotifyCompliance --> ReviewRequest

    ReviewRequest -->|ACCESS| AccessRequest
    ReviewRequest -->|CORRECTION| CorrectionRequest
    ReviewRequest -->|DELETION| DeletionRequest
    ReviewRequest -->|PORTABILITY| PortabilityRequest

    AccessRequest --> ExecuteAction
    CorrectionRequest --> ExecuteAction
    PortabilityRequest --> ExecuteAction

    DeletionRequest --> ValidateAction
    ValidateAction --> CheckRetention

    CheckRetention -->|No, Can Delete| ExecuteAction
    CheckRetention -->|Yes, Must Retain| CannotDelete

    CannotDelete --> NotifyUser

    ExecuteAction --> AuditLog2
    AuditLog2 --> NotifyUser
    NotifyUser --> UpdateRequest
    UpdateRequest --> Complete

    style AuditLog1 fill:#e1f5fe
    style AuditLog2 fill:#e1f5fe
    style CannotDelete fill:#fff3e0
    style Complete fill:#e8f5e9

10. Caching Strategy Flow

flowchart LR
    subgraph Request["API Request"]
        Client[Client Request]
        API[API Route]
    end

    subgraph Cache["Cache Layer"]
        CheckCache{Cache<br/>Hit?}
        Redis[(Redis<br/>Cache)]
    end

    subgraph Database["Database Layer"]
        Prisma[Prisma ORM]
        PG[(PostgreSQL)]
    end

    subgraph Response["Response Path"]
        Transform[Transform<br/>Data]
        Return[Return to<br/>Client]
    end

    Client --> API
    API --> CheckCache

    CheckCache -->|Hit| Transform
    CheckCache -->|Miss| Prisma

    Prisma --> PG
    PG --> Prisma

    Prisma --> Redis
    Prisma --> Transform

    Redis -.Cached for 5-60min.-> CheckCache

    Transform --> Return
    Return --> Client

    style CheckCache fill:#fff3e0
    style Redis fill:#ffe0b2
    style Transform fill:#e8f5e9

Cache Strategy:

Organization Data: 60 minutes (rarely changes)
User Permissions: 15 minutes (moderate changes)
Client List: 30 minutes (infrequent changes)
Campaign Data: 5 minutes (frequent updates during active campaigns)
Analytics: 10 minutes (balance between freshness and performance)
Templates: 60 minutes (static content)

Cache Invalidation:

Write operations invalidate related cache entries
Webhook events trigger cache invalidation
Manual cache clear available for admins

Data Flow Principles

1. Security First

All data flows through authentication layer
Authorization checks at every boundary
Encryption in transit (TLS 1.3)
Encryption at rest (PostgreSQL)

2. POPIA Compliance

Complete audit trail for all personal data operations
Consent verification before data processing
Data classification and handling based on sensitivity
South African data residency maintained

3. Performance Optimization

Redis caching for frequently accessed data
Efficient database queries with proper indexing
Async processing for non-critical operations
CDN for static assets

4. Multi-Tenant Isolation

Organization-based data partitioning
Row-level security in database queries
Separate cache namespaces per organization
No data leakage between tenants

5. Error Handling

Graceful degradation on service failures
Comprehensive error logging (no PII in logs)
User-friendly error messages
Automatic retry for transient failures

Performance Metrics

Target Response Times

Authentication Flow: <150ms
Campaign Creation: <300ms
Content Approval: <200ms
Analytics Dashboard Load: <800ms
Data Subject Request: <24 hours (POPIA requirement)
Audit Log Write: <50ms (async)

Data Volume Estimates

Audit Logs: ~10,000 entries/day per active organization
Analytics Events: ~50,000 events/day per active campaign
Cache Hit Rate: >80% for frequently accessed data
Database Query Time: <100ms (P95)

Last Updated: November 20, 2025 Verified: Production v0.1.4

Data Flow - ThriveSend B2B2G

Data Flow - ThriveSend B2B2G

Overview

1. Service Provider Onboarding Flow

2. Client Creation Flow (B2B2G)

3. Campaign Creation & Execution Flow

4. Content Approval Flow (Government)

5. User Authentication & Authorization Flow

6. POPIA Data Processing Flow

7. Analytics Data Collection Flow

8. Team Invitation Flow

9. Data Subject Request Flow (POPIA)

10. Caching Strategy Flow

Data Flow Principles

1. Security First

2. POPIA Compliance

3. Performance Optimization

4. Multi-Tenant Isolation

5. Error Handling

Performance Metrics

Target Response Times

Data Volume Estimates

Related Documentation