Skip to content

Customer / CRM (Customer 360)

Validation legend: ✅ verified in code · 📄 Excel-only · 💻 code-only · ♻️ duplicate/inconsistent · ⚰️ dead code · ⚠️ risk


1. Overview

Customer / CRM is the Customer 360 heart of Prism — the unified, omnichannel single view of every guest. It owns the customers collection: one deduplicated profile per (parent_business_id, mobileNo) carrying lifetime metrics (LTV, LTO, AOV), behavioural counters (day/meal/channel preferences), demographics (DOB, anniversary, gender), communication preferences (comm_array, DND/opt-in), loyalty state and ratings.

Three cooperating pieces make up the module: 1. customerController.js — the large (65+ exports) HTTP surface for reading, editing, exporting, notifying, and segmenting customers. The most central controller in the codebase. 2. appCustomerController.js + customerInjestionController.js — the async creation pipeline: the app writes raw customer records to a dump; a cron drains, deduplicates and upserts them into customers (and triggers welcome journeys). 3. dedupCustomers.js — a nightly safety net that collapses duplicate profiles by mobile number and re-points outlet records to the canonical customer.

Why it exists: orders arrive from many channels under the same phone number. Without one merged profile keyed on mobile, RFM, Campaigns and Loyalty would double-count and mistarget. Customer 360 is the unify half of Prism's "Ingest & Unify" layer; Order-Ingestion is the ingest half.

Business importance: Prism's core cross-widget integrity rule — Total Customers = Transacting + Non-Transacting and unique-customer counts — depends entirely on this module's dedup correctness (Product Overview §7).


2. Business Flow

2.1 Product view

A customer profile is created either implicitly (first order → Order-Ingestion upserts customers) or explicitly (app sign-up → dump → cron upsert). From then on, every order enriches the same profile: LTV/LTO/AOV recompute, day/meal/channel counters increment, lastOrderDate updates. The Marketing Manager and Analyst then read, filter, tag, export, and message customers through customerController.

2.2 Example journey (app sign-up → welcome → segment)

  1. Guest signs up in the brand app → POST /injestion/createCustomerappCustomerController.createCustomer writes {data:req.body, processed:false} to customer_injestion_dumps. Responds {status:1}. ✅
  2. customerInjestionController cron (every ~10 min) fetches up to 5000 {processed:false} dumps. ✅
  3. For each: dedup-check customers by (parent_business_id, mobileNo). If new, upsert with comm_array (["whatsapp","sms","email"] if valid email else ["whatsapp","sms"]). ✅
  4. If a welcome-message journey (campaign_journeys type 66dec4a99f76140e964ad2bb) is enabled, enqueue the welcome SMS/WhatsApp and debit the business wallet. ✅
  5. Overnight, RFM scores the profile and Customer-Intelligence filters can target it.
  6. Marketer opens Customer 360 (/get/customer/details), filters, tags (addEditCustomerTags), and exports or campaigns.

2.3 Edge cases

  • Duplicate profiles (created by dual-writer ingestion / races) → the nightly dedupCustomers cron merges them (canonical = highest LTO, then LTV, then smallest _id). ✅
  • Invalid emailcomm_array omits "email"; email_opened stays 0. ✅
  • DND / opt-outchangeDndFlag, unsubscribeCampaign, blacklistSegments exclude the customer from sends. ✅
  • Legacy per-tenant collections (customers_4580, customers_6300, customers_${businessId}) exist alongside the unified customers. ♻️
  • Collection-name drift: cron reads customer_injestion_dumps (plural) while models/customer_injestion_dump.js declares the singular model name. ♻️

3. Technical Flow

3.1 Async creation pipeline

App → POST /injestion/createCustomer   appCustomerController.createCustomer
        → customer_injestion_dumps {data, processed:false}                       ✅
App → POST /injestion/fcms             appCustomerController.createFcm
        → fcm_injestion_dump {data, processed:false}

crons/customerInjestionController.js  cron "*/10 * * * *" (Asia/Calcutta)          ✅
  ├─ stuck guard: cron_stats status:false > 30 min → force-close
  ├─ fetch customer_injestion_dumps {processed:false} limit 5000
  ├─ cron_stats.create {cron_data:"customerInjestion", status:false, batch}
  ├─ for each dump:
  │     lookup customers {parent_business_id, mobileNo} (hint mobileNo_1_parent_business_id_1_comm_array_1_gender_1)
  │     exists → mark dump processed:true
  │     new    → build comm_array (email? +email) → customers.updateOne(
  │                 {parent_business_id, mobileNo}, {$setOnInsert:createcustomer}, {upsert:true})
  │            → welcome journey? campaign_journeys → sendWelcomeMsg → sms_queue_*/whatsapp_queue_* + wallet debit
  ├─ customer_injestion_dumps.deleteMany({processed:true})
  └─ cron_stats.update {status:true, end_time}

3.2 Order-driven enrichment (from Order-Ingestion)

processCommonIngestion / per-POS processors upsert customers on every order: LTV += amount; LTO += 1; avg_order_size = LTV/LTO, update lastOrderDate, increment day/meal/channel counters (via parseOrderData). See Order-Ingestion §8.

3.3 Nightly dedup

crons/dedupCustomers.js  cron "0 1 * * *" (1 AM, Asia/Calcutta)                    ✅
  ├─ stuck guard: cron_stats status:false > 4 h → force-close (force_closed:true)
  ├─ pick businesses: business_configs {isParent:true, business_status:1, parent_business_id ≠ 5}
  ├─ per business, aggregate customers group by mobileNo where count>1
  ├─ pickCanonical(docs): sort LTO desc → LTV desc → _id asc → first
  ├─ re-point customers_outlets of non-canonicals to canonical
  │     (if canonical already covers that business_id → delete outlet; else re-point + mark covered)
  ├─ customers.deleteMany({_id ∈ nonCanonicalIds})
  └─ cron_stats {duplicatesFound, deleted, repointed, errors}
  Knobs: GROUP_BATCH=500, BIZ_SLEEP=50ms, BATCH_SLEEP=150ms, allowDiskUse:true

3.4 Read/act path (customerController)

route (authMiddleware) → customerController.<fn> → customers (+ business_configs, orders, ratings, wallet, MySQL) → response. Notifications fan out to FCM (Firebase Admin), SMS/WhatsApp queues, and S3 for exports.


4. Architecture Diagram

4.1 Module flowchart

flowchart TB
    subgraph create["Async creation"]
        APP[App sign-up] -->|POST /injestion/createCustomer| DUMP[(customer_injestion_dumps)]
        CIC{{customerInjestionController<br/>cron 10 min}} -->|drain + dedup| C[(customers)]
        DUMP --> CIC
        CIC -->|welcome journey| WQ[[sms/whatsapp queue]]
    end
    subgraph order["Order-driven enrichment"]
        OI[Order-Ingestion processors] -->|upsert LTV/LTO/AOV| C
        OI --> CO[(customers_outlets)]
    end
    subgraph dedup["Nightly safety net"]
        DD{{dedupCustomers cron 1 AM}} -->|merge by mobileNo| C
        DD -->|re-point| CO
    end
    subgraph read["Read / Act (customerController)"]
        UI[Marketer / Analyst / App] -->|authMiddleware| CC[customerController 65+ fns]
        CC --> C
        CC --> FCM[Firebase]
        CC --> S3[(S3 export)]
        CC --> RR[(rating_reviews)]
    end

4.2 Dedup sequence

sequenceDiagram
    autonumber
    participant CRON as dedupCustomers (1 AM)
    participant C as customers
    participant O as customers_outlets
    CRON->>C: aggregate group by mobileNo, count>1
    CRON->>CRON: pickCanonical (LTO↓, LTV↓, _id↑)
    CRON->>O: fetch canonical + non-canonical outlets
    alt canonical already covers business_id
        CRON->>O: delete duplicate outlet
    else
        CRON->>O: re-point outlet.customer_id → canonical
    end
    CRON->>C: deleteMany(non-canonical _ids)

5. Folder Structure

Layer File Role Validation
Controller controllers/customerController.js Core Customer 360 HTTP API (65+ exports) — largest controller
Controller controllers/appCustomerController.js createCustomer, createFcm → write dumps
Cron controllers/customerInjestionController.js drain customer_injestion_dumps → upsert customers + welcome journey
Cron crons/dedupCustomers.js nightly dedup by mobileNo + outlet re-point
Cron (legacy) controllers/oldCustomerCron.js superseded customer sync ⚰️
Route routes/crm.js /crm_api/get\|update/customer/*, ratings, notifications, segments
Route routes/injestion.js /injestion/createCustomer, /injestion/fcms (root mount)
Middleware middlewares/authMiddleware.js web/mobile token + auth_tokens multi-session + bcrypt fallback
Model models/customer_injestion_dump.js dump schema {data, processed} (singular ♻️ vs plural collection)
Model models/buisness_aggregator.js cohort rollups (active/lost/transacting counts)
Helper commonFunctions/sendSms.js, helpers.js SMS/WhatsApp send + wallet debit

Legacy customerController.customerInjestion / fcmInjestion (routed at /crm_api/customer/old/injestion, /fcm/old/injestion) use per-tenant customers_${businessId} collections and are superseded by the dump-cron flow. ⚰️♻️


6. Database

6.1 customers (the master) — inferred schema ✅

Field Type Notes
parent_business_id Number tenant key
mobileNo String(10) dedup key (with parent)
customer_name, email, gender String demographics
dob, dob_md, anniversary, anniversary_md Date/Number _md = MMDD for birthday/anniversary journeys
LTV, LTO, avg_order_size Number lifetime value / orders / AOV
lastOrderDate, createdAt Date/String recency inputs
day counters Number mon_count … sun_count
meal counters Number lunch_count, dinner_count, mid_night_count
channel counters Number swig_count, zom_count, pos_count, uen_count
comm_array Array e.g. ["whatsapp","sms","email"]
email_opened Number 0/1
rfm.{r,f,m,fm,sc,sg,wd,ca} Object written by RFM cron
milestone_level, segments Number/Array loyalty/segment state (implied)
fcms[] Array legacy per-tenant device list {fcm_id, os, status, fail_count}

Indexes (referenced via hints ✅): (parent_business_id, mobileNo); (parent_business_id, name text, mobileNo text); (parent_business_id, LTV desc, _id desc); (parent_business_id, avg_rating, LTV desc, _id desc).

Collection / table Store Purpose Key link
customers_outlets Mongo per-outlet customer record (customer_id, parent_business_id, business_id)
customer_injestion_dumps Mongo async creation buffer processed flag
fcm_injestion_dump Mongo FCM token buffer processed flag
fcm_tokens, fcm_tokens_${pid}, fcmjunks Mongo device tokens / junk mobile / os
rating_reviews, mis_ratings Mongo feedback + rollups mobile_number, parent_business_id
customer_communication Mongo SMS/email/push log mobile / campaign
campaign_journeys Mongo welcome/birthday journeys campaign_type_id
wallet_transactions, wallet_config Mongo wallet debit for sends business
business_configs Mongo tenant config parent_business_id, is_parent
buisness_aggregator Mongo cohort counts businessId
customers_${businessId} Mongo legacy per-tenant profiles ♻️ mobile
MySQL addo_contacts, addo_contacts_mapping, wallet, users, roles MySQL auth + legacy contacts/wallet contactId, mobileNo

Common queries:

// dedup-check on ingest
db.customers.findOne({ parent_business_id: 4580, mobileNo: "9876543210" })
// paginated segment read (LTV-sorted)
db.customers.find({ parent_business_id: 4580 }).sort({ LTV: -1, _id: -1 }).limit(50)
// find duplicate profiles for a tenant
db.customers.aggregate([{ $match:{ parent_business_id:4580 }},
  { $group:{ _id:"$mobileNo", n:{ $sum:1 }}},{ $match:{ n:{ $gt:1 }}}])

⚠️ Most MongoDB collections lack explicit indexes beyond customers (map-03 §5.1).


7. APIs

All under /crm_api (routes/crm.js), method POST unless noted; authMiddleware where marked. authMiddleware: header source (web/mobile) → validates token vs user apiToken / auth_tokens table (bcrypt fallback), sets req.body.created_by, scopes by business_id.

Route Handler Auth Purpose
/get/customer/details getCustomerDetails none fetch profile(s) — verify payload
/get/customer/detail getCustomerDetail authMiddleware single deep profile
/update/customer/details updateCustomerDetails authMiddleware edit demographics/DOB/anniversary
/update/customer/data updateCustomerData none bulk update
/add/manual/customer addManualCustomer (all) manual create
/change/customer/status changeCustomerStatus authMiddleware activate/deactivate
/blacklist/segments blacklistSegments authMiddleware exclude segments from sends
/save/customer_tags addEditCustomerTags authMiddleware tag customers
/get/unsubscribed/customers getUnsubscribedCustomers authMiddleware opt-out list
/download/customer(s) downloadCustomerData authMiddleware CSV/XLSX export (S3)
/get/order/list, /order/details getOrderList, orderDetails authMiddleware order history
/rating/reviews, /rating/mis ratingReviews, ratingMis mixed feedback
/allow_push_notification allow_push_notification authMiddleware push opt-in
/customer/intraction customerIntraction none log interaction
/gupshup/webhook/callback gupshupWebhook none WhatsApp inbound webhook
/injestion/createCustomer appCustomerController.createCustomer none (root mount) async create dump
/injestion/fcms appCustomerController.createFcm none (root mount) async FCM dump

Example — POST /crm_api/update/customer/details (verify):

{ "business_id": "4580", "mobileNo": "9876543210",
  "customer_name": "Asha", "email": "asha@x.com",
  "gender": "F", "dob": "1990-04-12", "anniversary": "2018-11-30" }
Response (typical): { "status": true, "message": "updated" }. Failure: { status:false, message }.

⚠️ Several customer-mutating routes (/get/customer/details, /update/customer/data, /add/manual/customer, /gupshup/webhook/callback) are registered without authMiddleware — a security review item (Security).


8. Code Walkthrough

customerController.js (65+ exports) — organised (informally) into clusters: - Retrieval: getCustomerDetails, getCustomerDetail, getCustomerList, getCustomerId, getCustomerInfo. - Mutation: updateCustomerDetails, updateCustomerData, addManualCustomer, changeCustomerStatus, changeCustomerCodStatus, addCustomerNotes, addWalletPoints. - Comms: SendIndividualNotification/Sms, sendWhatsapp, changeDndFlag, unsubscribeCampaign, getSenderId, getSmsCount. - Loyalty/rewards: feedbackLoyaltyPoints, getLoyaltyName, getCustomerRewards, customerRewardRedeem, validatePetpoojaOTP. - Segments/tags: blacklistSegments, addEditCustomerTags. - Import/export: uploadCustomerData, downloadCustomerData, downloadRatingReviews, downloadSalesData. - Legacy ingest: customerInjestion, fcmInjestion (per-tenant customers_${id} collections) ⚰️.

Touches 43+ collections and MySQL (addo_business, addo_contacts*, wallet, users, roles); external services FCM (fcm-push + firebase-admin), S3 (AWS.S3 v4, ap-south-1), SMS/WhatsApp via commonFunctions/sendSms + queues.

appCustomerController.createCustomer / createFcm — deliberately trivial: wrap req.body into a dump doc (processed:false) and return. All heavy lifting is deferred to the cron (keeps the app write path fast).

customerInjestionController cron — drains dumps, dedup-checks customers by (parent_business_id, mobileNo) using an index hint, upserts with $setOnInsert, builds comm_array from email validity, fires the welcome journey, then bulk-deletes processed dumps.

dedupCustomers cronpickCanonical() (LTO↓, LTV↓, _id↑) chooses the survivor; outlet records are re-pointed or deleted so customers_outlets stays consistent; non-canonical customers are deleted. Batched with sleeps and allowDiskUse to stay gentle on the shared web process.

Call hierarchy (creation): route → appCustomerController.createCustomer → customer_injestion_dumps(async)customerInjestionController cron → customers.upsert → sendWelcomeMsg → channel queues.


9. Business Rules

Rule Detail Validation
Dedup key (parent_business_id, mobileNo); mobileNo is 10-digit
Canonical selection LTO desc → LTV desc → _id asc
comm_array from email valid email ⇒ ["whatsapp","sms","email"], else ["whatsapp","sms"]
Welcome journey type campaign_journeys id 66dec4a99f76140e964ad2bb; debits wallet
Tenant scoping every query filters parent_business_id
Dedup tenant filter isParent:true, business_status:1, parent_business_id ≠ 5
Async creation app writes only to dump; cron does the real upsert
Stuck guards ingestion cron 30 min; dedup cron 4 h → force-close
Legacy per-tenant collections customers_${id} still exist alongside customers ♻️
Model/collection name drift model customer_injestion_dump vs collection customer_injestion_dumps ♻️
Unauthenticated mutating routes several customer routes lack authMiddleware ⚠️
Non-transacting customers profiles with no order exist (created via dump) → Total = Transacting + Non-Transacting ✅ / 📄

10. Performance

  • Async write path: app creation returns immediately (dump insert); heavy dedup/upsert batched in the cron (5000/run). ✅
  • Dedup throttling: GROUP_BATCH=500, per-business/per-batch sleeps, allowDiskUse:true keep the nightly merge from starving API traffic. ✅
  • Index hints: reads use compound indexes on (parent_business_id, mobileNo) / LTV-sorted variants. ✅
  • Bottlenecks ⚠️:
  • customerController is monolithic (65+ fns, 43+ collections) — a change-risk and cognitive hotspot.
  • Full-collection dedup aggregations grow with tenant size (unindexed on some collections).
  • Legacy customers_${id} collections fragment the customer base and complicate global queries.
  • All crons run in the web process (latency coupling — HLA §9).

11. Logging

  • Controllers: console.log + Winston (require("../logger")); errors returned as {status:false, message}.
  • Crons: cron_status rows are the durable trace (cron_data:"customerInjestion" / "dedupCustomers", start/end, batch, force_closed). Dedup writes {duplicatesFound, deleted, repointed, errors}.
  • Trace correlation: the cron_stats._id links a run to its work; processed:true on a dump marks completion.
  • Interaction/audit: intraction_logs, user_data_logs, customer_update_logs, business_management_logs capture edits.

12. Monitoring

Signal Where Healthy
Dump backlog customer_injestion_dumps.countDocuments({processed:false}) small, draining every 10 min
Creation cron cron_status newest customerInjestion status:true, fresh end_time
Dedup run cron_status newest dedupCustomers daily status:true; low errors
Duplicate rate dedup duplicatesFound trend stable/declining
Cohort integrity buisness_aggregator (transacting vs total) Total = Transacting + Non-Transacting holds
Comms consistency comm_array distribution matches opt-in/DND state

"Healthy" = dump backlog near zero, both crons green daily, duplicatesFound low and not trending up, cohort totals satisfy the integrity rule.


13. Troubleshooting

Symptom Root cause Resolution / commands
New app sign-ups not in Customer 360 creation cron stuck / dump not draining db.cron_status.find({cron_data:"customerInjestion"}).sort({_id:-1}).limit(1); check db.customer_injestion_dumps.countDocuments({processed:false}); restart PM2
Two profiles for one phone duplicate created before dedup ran (dual-writer ingestion / race) run/inspect dedupCustomers; db.customers.aggregate(... group mobileNo count>1); verify canonical merge
Customer counts don't reconcile leftover customers_${id} legacy docs / partial dedup audit legacy collections; confirm dedup tenant filter includes the tenant
Welcome message not sent journey disabled / wallet insufficient check campaign_journeys type 66dec4a99f76140e964ad2bb enabled; wallet balance
Export fails S3 creds/region or large dataset check AWS.S3 config (ap-south-1, v4); paginate
Unauthorized edits route missing authMiddleware apply auth (see Security)

14. FAQs

  • What's the customer primary key? (parent_business_id, mobileNo) — mobile is 10-digit, tenant-scoped. There is no single global customer id.
  • When is a profile created? On first order (ingestion upsert) or app sign-up (dump → cron). Both dedup by mobile.
  • How is the canonical duplicate chosen? Highest LTO, then LTV, then smallest _id.
  • Why a dump + cron instead of direct write? To keep the app write path fast and to funnel dedup/journey logic through one place.
  • What are customers_outlets for? Per-outlet view of a customer (a chain guest can transact at many outlets under one canonical profile).
  • Why do customers_4580 / customers_6300 exist? Legacy per-tenant collections from before unification — being phased out. ♻️
  • Where do LTV/LTO come from? Recomputed by Order-Ingestion processors on every order.

15. Cheat Sheet

MASTER:   customers  key (parent_business_id, mobileNo)  fields LTV/LTO/avg_order_size/comm_array/rfm.*
OUTLETS:  customers_outlets  key (customer_id, parent_business_id, business_id)
CREATE:   POST /injestion/createCustomer → customer_injestion_dumps {processed:false}
          cron customerInjestionController "*/10 * * * *" → upsert customers + welcome journey (wallet debit)
DEDUP:    crons/dedupCustomers.js  "0 1 * * *"  canonical = LTO↓, LTV↓, _id↑  → re-point outlets, delete dups
          tenant filter: isParent:true, business_status:1, parent_business_id≠5 ; stuck 4h → force-close
READ/ACT: controllers/customerController.js (65+ fns) via /crm_api + authMiddleware
          getCustomerDetails/updateCustomerDetails/blacklistSegments/addEditCustomerTags/downloadCustomerData
COMMS:    comm_array = email? [wa,sms,email] : [wa,sms] ; DND via changeDndFlag/unsubscribeCampaign
EXT:      FCM(firebase-admin) | SMS/WhatsApp queues | S3(ap-south-1) | MySQL(addo_contacts*, wallet, users)
GOTCHAS:  legacy customers_${id} ♻️ | dump model singular vs collection plural ♻️ | some routes no authMiddleware ⚠️
          | dedup depends on accurate LTO/LTV | mobile is the only identity


Knowledge Tests

Level 1 — MCQs

  1. The customer dedup key is: a) email b) _id c) (parent_business_id, mobileNo) d) order_idc ✅.
  2. dedupCustomers chooses the canonical by: a) newest _id b) LTO↓, then LTV↓, then _id c) alphabetical name d) most outlets — b.
  3. App sign-up writes directly to customers? a) yes b) no — to customer_injestion_dumps, drained by a cron c) to Redis d) to MySQL — b.
  4. comm_array for a customer with no valid email is: a) [] b) ["email"] c) ["whatsapp","sms"] d) ["push"]c.
  5. Which cron force-closes after 4 hours stuck? a) customerInjestion (30 min) b) dedupCustomers c) rfmSegments d) none — b.

Level 2 — Scenarios

  1. A chain sees Total Customers ≠ Transacting + Non-Transacting. You suspect duplicate or legacy profiles. Describe how you'd confirm using dedupCustomers output and the legacy customers_${id} collections, and how to remediate.
  2. Welcome messages stopped for new app users of one brand, though profiles are being created. Where do you look (journey config, wallet, queue) and in what order?

Level 3 — Code reading

Open crons/dedupCustomers.js. Explain the outlet re-pointing branch: when is a non-canonical outlet deleted vs re-pointed, and why the coveredBizIds set is updated as it goes.

Level 4 — Architecture

customerController.js is a 65-function, 43-collection monolith. Propose a decomposition into cohesive sub-modules (retrieval / mutation / comms / import-export / legacy) without breaking the /crm_api route contract. Address shared helpers and the legacy customers_${id} collections.

Level 5 — Production debugging

The customerInjestion cron shows status:false for 40 minutes; customer_injestion_dumps {processed:false} is climbing and welcome messages have stalled. Diagnose the stuck run (guard is 30 min → should have force-closed; if not, the current run is throwing mid-batch), give the immediate fix and a structural safeguard.


Practical Assignments

  1. Trace a sign-up end-to-end: POST to /injestion/createCustomer on a dev tenant, watch the dump appear (processed:false), the cron upsert into customers, the welcome message enqueue, and the dump get deleted. Record timings for each hop.
  2. Reproduce dedup: manually insert two customers docs with the same mobileNo (different LTO/LTV) plus outlet records, run dedupCustomers logic, and verify the canonical survives with outlets re-pointed.
  3. Harden a route: pick one unauthenticated mutating customer route, add authMiddleware, and document the before/after request contract and any callers you must update.