Skip to content

Automated-Journeys Module

Validation legend: ✅ verified in code · 📄 from business/PM source · 💻 verified this review · ♻️ duplicate/overlapping implementation · ⚰️ dead/legacy · ⚠️ needs verification / open bug

⚠️ E1.7 (P0, open): the PM roadmap states "All 8 automated journey logics need verification." This review confirms the concern in code — only 4 of 8 journeys are handled in automatedCampaignQueueNew, 1 is a separate cron, 1 is partial, and 2 (Low Rating Rewards, Referral Boost) were not found in the queue builder at all. See §2 and §8.


1. Overview

What: Automated Journeys ("Auto Campaigns", being renamed Journeys per Roadmap B3.6) are event-triggered outreach automations following the EVENT → CONDITION → ACTION model. A client configures a journey once; Prism then runs it automatically forever — sending WhatsApp/SMS/Push to customers who match the trigger.

Why it matters: Journeys are the always-on retention engine that moves guests up the RFM ladder (📄 Business-Flow §3-4) without manual effort. They share the exact same channel queues, wallet-debit path, and ROI attribution crons as manual Campaigns — Roadmap A2.1 (P0) merges the two into a Central Marketing Hub.

The 8 business journeys (📄 Business-Flow) and their verified code status (💻):

# Journey EVENT Code status Where
1 Welcome Sign-up ⚠️ PARTIAL — id 66dec4a9… referenced for ROI, not in queue switch welcomeMsgRoi.js
2 Birthday Birthday approaching ✅ ACTIVE — 627b8906… automatedCampaignQueueNew
3 Anniversary Anniversary ✅ ACTIVE — 627b891f… automatedCampaignQueueNew
4 Wallet Expiry Points near expiry ✅ ACTIVE — 627b8b94… automatedCampaignQueueNew
5 Lost Customers No order for X days ✅ ACTIVE — 627b8c4b… automatedCampaignQueueNew
6 Abandoned Cart Cart abandoned X min ✅ ACTIVE — 670cef0a… separate abandonCartCron.js (*/5)
7 Low Rating Rewards Low rating submitted ⚠️ NOT FOUND in queue builder
8 Referral Boost Order-count threshold ⚠️ NOT FOUND in queue builder

This table alone is the reason E1.7 exists. The docs below describe the intended 8-journey design and the actual code.


2. Business Flow (product view)

flowchart TD
    E1[Sign-up] --> W[Welcome]
    E2[Birthday approaching] --> B[Birthday]
    E3[Anniversary] --> AN[Anniversary]
    E4[Points near expiry] --> WE[Wallet Expiry]
    E5[No order for X days] --> LC[Lost Customers]
    E6[Cart abandoned X min] --> AC[Abandoned Cart]
    E7[Low rating submitted] --> LR[Low Rating Rewards]
    E8[Order-count threshold] --> RB[Referral Boost]
    W & B & AN & WE & LC & AC & LR & RB --> CH{{Channel: WhatsApp / SMS / Push}}
    CH --> M[auto_campaign_roi: delivery, orders, revenue]

One-time setup, then automatic. On activation, up to 10,000 eligible past customers are backtracked (📄 Business-Flow §4 — the "10K cap"). ⚠️ The literal 10000 is not a constant in automatedCampaignQueueNew; the builder paginates the audience in batches of 1000 and honors an optional per-journey daily_cap. The 10K cap is a business/product rule to confirm against config, not a hardcoded literal (verify).

Example journey — Birthday: 1. Client enables "Birthday" journey, picks WhatsApp channel + a birthday_msg template with a coupon, sets send time. 2. saveCampaignJourney writes config into campaign_journeys.campaign_data; enableAutoMatedCampaign sets campaign_enabled:1. 3. Nightly at 10 PM IST, processAutomatedCampaigns finds journeys with campaign_enabled:1 AND config_enabled:1 and calls automatedCampaignQueueNew(journey). 4. The builder queries wallet/customers for those whose birthday matches the send window, replaces ue_cust_name/ue_cust_mobile/ue_cust_eid, deducts wallet, and enqueues into whatsapp_queue_YYYYMMDD (or sms_queue_*/push_noti_queue_*). 5. Channel send crons deliver; autoCampaignRoi.js (6 AM) attributes any resulting orders → auto_campaign_roi.

Edge cases: - Abandoned Cart runs on a different cadence — every 5 min from MySQL addo_orders, not the 10 PM batch. Same-day dedup via auto_campaign_roi. - daily_cap ✅ — Math.min(eligibleCustomers, daily_cap) throttles per-journey volume. - Insufficient wallet ✅ — balance checked before queueing; customer skipped. - Journey shows inactive in UI — the autoCampaign.js cron (12:47 PM) syncs business_config.auto_campaign from journey state; if it hasn't run, UI status can lag.


3. Technical Flow

sequenceDiagram
    autonumber
    participant UI as CRM Frontend
    participant API as /crm_api (routes/crm.js)
    participant CTL as automatedCampaignController.saveCampaignJourney
    participant CJ as campaign_journeys (Mongo)
    participant CRON as processAutomatedCampaigns (0 22 * * *)
    participant QB as automatedCampaignQueueNew(journey)
    participant WAL as deductBalance / wallet
    participant Q as sms/whatsapp/push queue_YYYYMMDD
    participant SND as channel send crons
    participant ROI as autoCampaignRoiCrons (early AM)

    UI->>API: POST /save/campaign/journey (auth)
    API->>CTL: config + channels + templates + thresholds
    CTL->>CJ: upsert campaign_data; set business_config.auto_campaign
    Note over CRON: nightly 10 PM IST
    CRON->>CJ: find {campaign_enabled:1, config_enabled:1}
    loop each journey
      CRON->>QB: automatedCampaignQueueNew(journey)
      QB->>QB: switch on campaign_type_id (Birthday/Anniv/Lost/WalletExpiry)
      QB->>WAL: check + deduct wallet
      QB->>Q: insert per-customer rows (batch 1000, honor daily_cap)
    end
    SND->>Q: poll & send
    ROI->>CJ: attribute orders → auto_campaign_roi

Abandoned Cart bypasses the nightly path:

flowchart LR
    A[addo_orders MySQL<br/>status=0, total>1, os in A/I/website] --> B[abandonCartCron */5]
    B --> C{updatedAt in<br/>trigger window?}
    C -->|yes, not sent today| D[sendAbandonCartMsg]
    D --> E[sms_queue_* / whatsapp_queue_*]
    D --> F[auto_campaign_roi + wallet_transactions]

4. Architecture Diagram

flowchart TD
    subgraph Config[Configuration - HTTP]
      SC[saveCampaignJourney]
      EN[enableAutoMatedCampaign]
    end
    SC --> CJ[(campaign_journeys)]
    EN --> CJ
    SC --> BC[(business_config.auto_campaign)]
    CT[(campaign_types)] --> SC
    subgraph Scheduling[Crons]
      P1[processAutomatedCampaigns<br/>0 22 * * *]
      P2[autoCampaign<br/>47 12 * * *]
      P3[abandonCartCron<br/>*/5]
      P4[dripCampaignCron<br/>0 12 * * 1]
    end
    CJ --> P1
    P1 --> QB[automatedCampaignQueueNew]
    QB --> SQ[(sms_queue_*)]
    QB --> WQ[(whatsapp_queue_*)]
    QB --> PQ[(push_noti_queue_*)]
    P3 --> SQ
    P3 --> WQ
    CJ --> P2 --> BC
    QB -.legacy.-> LEG[automatedCampaignQueue ♻️]
    subgraph ROI[autoCampaignRoiCrons]
      R1[autoCampaignRoi 6AM]
      R2[abandonCartRoi 6:30]
      R3[welcomeMsgRoi 7:30]
      R4[lostCustomersRoi 0:00]
      R5[referEarnRoi 5:30]
    end
    CJ --> ROI --> ACR[(auto_campaign_roi)]

5. Folder Structure

uengage-crm/
├── controllers/
│   └── automatedCampaignController.js   ✅ config CRUD + ROI views (13 fns)
├── commonFunctions/
│   ├── automatedCampaignQueueNew.js     ✅ MAIN builder (~94KB)
│   ├── automatedCampaignQueue.js        ♻️ legacy builder (~73KB, uses customers_{business_id})
│   ├── smsCampaignQueue.js              ✅ insert wrapper → sms_queue_{tomorrow}
│   ├── pushCampaignQueue.js             ✅ insert wrapper → push_noti_queue_{tomorrow}
│   └── automatedCampaignWalletTransaction.js  ✅ deductBalance (per-message)
├── crons/
│   ├── processAutomatedCampaigns.js     ✅ 0 22 * * *  → automatedCampaignQueueNew
│   ├── autoCampaign.js                  ✅ 47 12 * * * → sync auto_campaign flag
│   ├── abandonCartCron.js               ✅ */5  (journey 6, separate)
│   ├── dripCampaignCron.js              ✅ 0 12 * * 1 (reports, NOT sends)
│   └── autoCampaignRoiCrons/*           ✅ ROI attribution (see §8)
├── models/
│   ├── campaign_journey.js              ✅ collection: campaign_journey
│   └── campaign_types.js                ✅ collection: campaign_type
└── routes/crm.js                        ✅ base /crm_api

6. Database

Collection Purpose Key fields Index Relationships
campaign_journey / campaign_journeys Per-business journey config + rules. Cron queries campaign_enabled:1, config_enabled:1. parent_business_id, child_business_id, is_parent, campaign_type_id (ObjectId), run_status (default true), campaign_data (Mixed: channels, templates, thresholds, daily_cap), campaign_enabled, config_enabled ⚠️ none — add (campaign_enabled, config_enabled) many→1 campaign_type
campaign_type Master catalog of journey types + dynamic form field schema. title, description, campaign_type, icon_url, execution_type (enum '1'/'2'), fields (array) none referenced by campaign_journey.campaign_type_id
campaign_fields Dynamic form field definitions referenced by config. ref_ids, options ⚠️ used in getFieldParameters
auto_campaign_roi Per-journey delivery + ROI rows (delivered, orders count, order amount). campaign_journey ref, mode (sms/push/whatsapp), delivered, orders, revenue, date ⚠️ ROI reporting + same-day dedup
addo_campaigns Journey-generated send records (history view via viewSentCampaigns). campaign_journey_id, businessId, parentBusinessId, sent_count, completed_at ⚠️ 1 journey→many sends
wallet_transaction Debit per message (via deductBalance). transaction_type:"debit", auto_id (campaign id), amounts none billing

Queues written: sms_queue_YYYYMMDD, whatsapp_queue_YYYYMMDD (.add(1,'days') in builder), push_noti_queue_YYYYMMDD. See SMS-Push / Queues.

Common queries:

// Nightly pickup (verified)
db.campaign_journeys.find({ campaign_enabled: 1, config_enabled: 1 })

// Abandoned cart source (verified, MySQL)
SELECT ... FROM addo_orders ao
 WHERE status=0 AND total>1 AND os IN ('A','I','website')
   AND (date(abandoned_cart_waba)!=CURDATE() OR abandoned_cart_waba IS NULL)
   AND ao.updatedAt BETWEEN NOW()-INTERVAL :end MINUTE AND NOW()-INTERVAL :start MINUTE

// ROI users
db.auto_campaign_roi.find({ campaign_journey_id: id, mode: "whatsapp" })


7. APIs

Base /crm_api, controller automatedCampaignController (routes/crm.js), all authMiddleware unless noted. ✅ verified mounted; request/response bodies ⚠️ verify-in-controller.

Method Path Fn Purpose
POST /crm_api/autoCampaign createCampaignType Create a journey-type master record
POST /crm_api/get/campaign/type/ids getCampaignType List journey types (+ ROI snapshot)
POST /crm_api/get/campaign/fields getCampaignField Field schema for a type
POST /crm_api/get/field/parameters getFieldParameters Dynamic dropdown options
POST /crm_api/save/campaign/journey saveCampaignJourney Save journey config (channels, templates, thresholds)
POST /crm_api/enable/automated/campaign/ enableAutoMatedCampaign Toggle campaign_enabled 0/1
POST /crm_api/get/campaign/mapped/business getCampaignMappedBusiness Configs per business + ROI metrics
POST /crm_api/get/auto_campaign_users getAutomatedCampaignUsers Paginated ROI user list
POST /crm_api/view/campaign/journey viewCapmaignJourney Full config for editing
POST /crm_api/get/days/type/options getDaysTypeOption Send-timing options (6 fixed field ids)
POST /crm_api/view/sent/automated/campaigns viewSentCampaigns Journey send history
POST /crm_api/automated/campaigns/report autoReports wallet_transactions audit by auto_id

Example payload (representative — ⚠️ verify against saveCampaignJourney)

POST /crm_api/save/campaign/journey
Authorization: <api-token>

{
  "parent_business_id": "123",
  "child_business_id": "456",
  "campaign_type_id": "627b890636d8cb4ebca28c0e",   // Birthday
  "campaign_data": {
    "channels": ["whatsapp","sms"],
    "whatsapp_template": "birthday_msg",
    "daily_cap": 500,
    "send_days_type": "on_birthday"
  }
}
{ "status": true, "message": "Journey saved", "campaign_journey_id": "665f..." }

Failure cases: 401/403 fail-closed; invalid campaign_type_id ref → validation error; disabling sets campaign_enabled:0 so the nightly cron skips it.


8. Code Walkthrough

crons/processAutomatedCampaigns.js ✅ (💻): "0 22 * * *" IST. Queries campaign_journeys {campaign_enabled:1, config_enabled:1}, loops and calls automatedCampaignQueueNew(journey) per record. No LIMIT / no 10000 cap here — every enabled journey is processed each night.

commonFunctions/automatedCampaignQueueNew.js ✅ (💻, the builder): a switch on the journey's campaign_type_id handles: - Birthday 627b8906…birthday_msg - Anniversary 627b891f…anniversary_template - Lost Customers 627b8c4b…lost_customers - Wallet Expiry 627b8b94… (wallet query LIMIT 1000)

For each: audience pulled from wallet/customers in batches of 1000 (cursor pagination), optional daily_cap via Math.min(eligible, daily_cap), wallet checked/deducted via deductBalance, template variables ue_cust_name / ue_cust_mobile / ue_cust_eid substituted (WhatsApp buttons get JWT-signed tracking URLs via update_template()), then rows inserted into the channel queue. Welcome, Low Rating Rewards, and Referral Boost are NOT handled in this switch (💻 — direct evidence for E1.7).

commonFunctions/automatedCampaignQueue.js ♻️ (legacy, ~73KB): same four journeys but reads from per-outlet customers_{business_id} collections and uses fixed skip/limit loops instead of cursor pagination. Not imported by processAutomatedCampaigns (which uses the New version) → likely dead/legacy for the nightly path; verify no other cron still calls it.

crons/abandonCartCron.js ✅ (💻): "*/5 * * * *", journey id 670cef0a…. Reads MySQL addo_orders (status=0, total>1, os in A/I/website, within abandon_cart_trigger window, default 30 min), dedups per-day via auto_campaign_roi, builds SMS (sendAbandonCartMsg) or WhatsApp (Gupshup), deducts wallet, enqueues to sms_queue_*/whatsapp_queue_*, writes auto_campaign_roi + wallet_transactions.

crons/autoCampaign.js ✅ (💻): "47 12 * * *". For each parent→child, checks campaign_journeys {campaign_enabled:1, config_enabled:1}; sets business_config.auto_campaign = 1 if any, else 0 (bulkWrite). Drives the UI "active" badge.

crons/dripCampaignCron.js ✅ (💻): "0 12 * * 1" (Mondays). Not a sender — generates 5 XLSX health reports (active/inactive/digital-bill/auto-campaign/feedback-flow users) to /public/dripCampaignReport/ and emails them internally.

Call hierarchy:

saveCampaignJourney / enableAutoMatedCampaign → campaign_journeys (+ business_config.auto_campaign)
processAutomatedCampaigns (0 22)
  └─ automatedCampaignQueueNew(journey)
       ├─ switch(campaign_type_id): Birthday | Anniversary | Lost | WalletExpiry
       ├─ deductBalance()  (automatedCampaignWalletTransaction)
       ├─ update_template() (WA JWT button URLs)
       └─ → sms_queue_* / whatsapp_queue_* / push_noti_queue_*
abandonCartCron (*/5)  → sendAbandonCartMsg → queues + auto_campaign_roi
autoCampaign (47 12)   → business_config.auto_campaign
autoCampaignRoiCrons/* (early AM) → auto_campaign_roi


9. Business Rules

  • 8 journeys designed, ~4.5 implemented 💻 — Birthday/Anniversary/Lost/Wallet-Expiry (queue), Abandoned-Cart (separate cron), Welcome (ROI only), Low-Rating-Rewards & Referral-Boost missingE1.7 (P0).
  • 10K backtracking cap 📄 — product rule on activation; code paginates in 1000-row batches and does not contain a literal 10000 — ⚠️ verify where the cap is enforced (config vs upstream).
  • daily_cap ✅ — per-journey throttle, Math.min(eligible, daily_cap).
  • Nightly batch at 10 PM IST ✅ — all non-cart journeys fire once/day; Abandoned Cart is near-real-time (*/5).
  • Wallet debit per message ✅ — via deductBalance; skips customer if balance insufficient.
  • Dedup ✅ — Abandoned Cart uses abandoned_cart_waba (MySQL) + auto_campaign_roi (same-day) to avoid re-sending.
  • Template variables ✅ — ue_cust_name, ue_cust_mobile, ue_cust_eid; WhatsApp buttons carry JWT tracking URLs.
  • UI active flag ✅ — synced daily by autoCampaign.js (12:47) from campaign_enabled+config_enabled.
  • Duplicate builders ♻️ — automatedCampaignQueue.js (legacy) vs New — tech-debt, consolidate (Roadmap, Tech-Debt).

10. Performance

  • Nightly spike — all enabled journeys processed in one 10 PM run with no LIMIT; large tenants create a burst of queue inserts + wallet queries (each journey scans wallet in 1000-row pages).
  • Abandoned Cart */5 hits MySQL addo_orders every 5 min — needs (status, updatedAt, os) index (⚠️).
  • Downstream drain governed by channel crons: SMS */3 (batch 8000), Push */5, WhatsApp */3s+*/1s.
  • Bottleneck: synchronous per-journey audience expansion in a single nightly process; no fan-out/parallelism.

11. Logging

  • Crons log lifecycle: "auto campaign cron started/ended", "cron started abandon cart".
  • Money trail: wallet_transaction (auto_id). Delivery/ROI: auto_campaign_roi. Sends history: addo_campaigns (campaign_journey_id, sent_count, completed_at).
  • No per-request trace_id in crons (that is HTTP-only) — correlate by campaign_journey_id + date.

12. Monitoring

  • Did tonight's run happen? Check cron_status for the 10 PM processAutomatedCampaigns entry; grep logs for "auto campaign cron ended".
  • Queue depth after 10 PM: db["whatsapp_queue_"+YYYYMMDD].countDocuments({process_status:0}), sms_queue_*/push_noti_queue_* {processed:false}.
  • ROI freshness: latest auto_campaign_roi rows per journey; ROI crons run 0:00–7:30.
  • UI badge wrong: confirm autoCampaign.js (12:47) ran and business_config.auto_campaign matches journey state.

13. Troubleshooting

Issue Root cause Resolution Command
Journey enabled but nothing sent config_enabled != 1, or 10 PM cron didn't run Verify both flags; check cron_status db.campaign_journeys.find({_id:ObjectId(id)},{campaign_enabled:1,config_enabled:1})
Low-rating / referral journey never fires ⚠️ not implemented in builder (E1.7) Confirm gap; escalate inspect switch in automatedCampaignQueueNew.js
Abandoned cart double-sent dedup flag not set Check abandoned_cart_waba + auto_campaign_roi SELECT abandoned_cart_waba FROM addo_orders WHERE id=?
UI shows journey inactive autoCampaign.js lagging Wait for/trigger 12:47 sync db.business_configs.find({...},{auto_campaign:1})
ROI zero despite sends ROI cron window / mismatch Check auto_campaign_roi + relevant ROI cron inspect crons/autoCampaignRoiCrons/*
Sends stopped mid-run wallet exhausted Top up wallet; check skips db.wallet_transaction.find({auto_id:id})

14. FAQs

  • When do journeys send? All except Abandoned Cart at 10 PM IST nightly; Abandoned Cart every 5 min.
  • Are all 8 journeys live? No — only 4 in the queue builder + Abandoned Cart cron; Welcome partial; Low-Rating & Referral-Boost missing (E1.7).
  • Where's the 10K cap in code? Not a literal — audience is paged 1000 at a time; the 10K is a product rule (verify enforcement).
  • New vs legacy queue file? automatedCampaignQueueNew.js is used by the nightly cron; automatedCampaignQueue.js is legacy ♻️.
  • Do journeys cost money? Yes — same wallet debit as manual campaigns.

15. Cheat Sheet

Config: campaign_journeys (campaign_enabled + config_enabled)   types: campaign_type
Nightly: processAutomatedCampaigns  "0 22 * * *"  → automatedCampaignQueueNew
  switch: Birthday 627b8906 | Anniversary 627b891f | Lost 627b8c4b | WalletExpiry 627b8b94
Abandoned Cart: abandonCartCron "*/5"  (id 670cef0a, MySQL addo_orders)
Flag sync: autoCampaign "47 12 * * *" → business_config.auto_campaign
Reports (not sends): dripCampaignCron "0 12 * * 1"
ROI: autoCampaignRoi 6:00 · abandonCartRoi 6:30 · welcomeMsgRoi 7:30 · lostCustomersRoi 0:00 · referEarnRoi 5:30 → auto_campaign_roi
Audience batch: 1000 · daily_cap = Math.min(eligible, cap)
Vars: ue_cust_name / ue_cust_mobile / ue_cust_eid
Queues: sms_queue_* whatsapp_queue_*(tomorrow) push_noti_queue_*
Legacy ♻️: automatedCampaignQueue.js   MISSING journeys: Low-Rating, Referral-Boost (E1.7)


Knowledge Tests

Level 1 — MCQs

  1. When does the main journey cron run? A) */5 B) 47 12 * * * C) 0 22 * * * D) 0 12 * * 1C (10 PM IST).
  2. Which journey runs on a separate 5-min cron? A) Birthday B) Abandoned Cart C) Anniversary D) Wallet Expiry → B.
  3. Which two journeys are NOT found in the queue builder? A) Birthday+Anniversary B) Welcome+Lost C) Low-Rating+Referral-Boost D) Wallet-Expiry+Abandoned-Cart → C (E1.7).
  4. What flags must be set for the nightly cron to pick a journey? A) run_status B) campaign_enabled:1 && config_enabled:1 C) status:0 D) is_parentB.
  5. Audience is paged in batches of… A) 8000 B) 5000 C) 1000 D) 10000 → C.

Level 2 — Scenarios

  1. A client enabled the "Low Rating Rewards" journey a month ago; nothing has ever sent. Explain the most likely root cause and how you'd confirm it. (Not implemented in automatedCampaignQueueNew switch — E1.7; confirm by reading the switch.)
  2. Birthday messages went out at ~10:15 PM but the client expected 9 AM. Explain the architecture reason and where send-time is honored. (Nightly builder enqueues at 10 PM with an execution_time/send-time from config; channel cron delivers when execution_time <= now.)

Level 3 — Code Reading

Open commonFunctions/automatedCampaignQueueNew.js. (a) List the campaign_type_id values handled in the switch and their template codes. (b) Where is daily_cap applied and with what expression? (c) Which three template variables are replaced for SMS?

Level 4 — Architecture

The nightly cron processes every enabled journey serially with no LIMIT. Design a fan-out that parallelizes per parent-business while preserving wallet-balance correctness and the daily_cap guarantee. What new failure modes appear?

Level 5 — Production Debugging

Abandoned-cart WhatsApp messages are being sent twice to some customers on the same day. Investigate: (1) the MySQL dedup (abandoned_cart_waba), (2) the auto_campaign_roi same-day check, (3) whether two cron instances overlap (*/5 with a long run), (4) timezone boundary at CURDATE(). Provide the SQL/Mongo you'd run and the fix.


Practical Assignments

  1. Audit journey coverage: map each of the 8 business journeys to its exact code path (or "missing"), citing file+line. Produce the E1.7 gap list.
  2. Trace a Birthday send: enable a test Birthday journey, run/inspect processAutomatedCampaigns output, and follow one customer row from campaign_journeysautomatedCampaignQueueNewwhatsapp_queue_*auto_campaign_roi.
  3. Propose the missing journeys: write a design (config schema + builder switch case + ROI cron) for "Low Rating Rewards", reusing the Birthday pattern. Do not implement in prod.