Author SHA1 Message Date
dependabot[bot] b8a831b3cf Bump mongoose from 8.9.3 to 8.9.5
Bumps [mongoose](https://github.com/Automattic/mongoose) from 8.9.3 to 8.9.5.
- [Release notes](https://github.com/Automattic/mongoose/releases)
- [Changelog](https://github.com/Automattic/mongoose/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Automattic/mongoose/compare/8.9.3...8.9.5)

---
updated-dependencies:
- dependency-name: mongoose
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-17 06:49:08 +00:00
2 changed files with 27 additions and 90 deletions
+24 -86
View File
@@ -1,91 +1,29 @@
# 📢 Distributed Notification and Alert System
# Notification System
## 📌 Overview
## Description
This project is a distributed notification system designed to process and deliver notifications to users through various channels (email, SMS, push notifications). The system ingests notifications, processes them based on user preferences and various business rules, and then delivers them to the appropriate channel.
This project is a **distributed backend system** designed to process and deliver notifications across multiple channels (Email, SMS, Push). It supports **real-time alerts, scheduled notifications, user preference handling, and query-driven delivery logic**. The system is built for scalability, resilience, and extensibility, leveraging **Node.js, Kafka, MongoDB, and Dockerized microservices**.
## Implementation Details
- **Backend Framework**: The system is built with **Node.js** and **Express.js**.
- **Database**: **MongoDB** is used for storing user preferences, notifications, and analytics data.
- **Kafka**: Kafka is used as the messaging system to handle notification ingestion and processing.
- **Job Scheduling**: **node-schedule** is used for scheduling notification delivery and periodic checks.
- **Mongoose**: Used for interacting with MongoDB.
---
## Design Choices
1. **Kafka for Decoupling**: Kafka is used to decouple the ingestion, processing, and delivery of notifications. This ensures that notifications can be processed independently and asynchronously, allowing for better scalability and resilience.
2. **MongoDB**: Chose MongoDB due to its flexibility in handling semi-structured data, especially for user preferences and notification statuses.
3. **User Preferences**: Users can configure preferences for receiving notifications through channels such as email, SMS, and push notifications. Additionally, they can define quiet hours to avoid receiving notifications during specific periods.
4. **Notification Throttling and Deduplication**: A throttling mechanism ensures that users do not receive too many notifications in a short period. Deduplication prevents sending the same notification more than once within a specific time window.
5. **Priority-Based Delivery**: Notifications are prioritized, with high-priority notifications being sent immediately, while low-priority notifications are aggregated and sent later.
## 🎯 Key Features
## Known Issues
1. **Error Handling in Kafka Consumer**: Error handling for Kafka consumer is not yet fully implemented. If Kafka is unavailable or there's a consumer issue, it might lead to failures in message consumption.
2. **Notification Delivery Failure Handling**: In some cases, if the notification fails to be sent via any channel, it will be marked as 'failed'. The retry logic for failed notifications is not yet implemented.
3. **Scaling Issues**: The current implementation is not optimized for scaling, as all services are running in a single instance without load balancing.
4. **Notification Status Issue**: When sending a notification with **normal** or **low priority** and a **future send time**, the notification should be saved with a **pending** status in the database. However, it is incorrectly being saved with a **sent** status instead. This issue needs to be addressed to ensure proper notification status handling before delivery.
- **RealTime & Scheduled Delivery**: Immediate processing for urgent alerts, scheduled pipelines for deferred notifications.
- **User Preferences**: Configurable quiet hours, channel selection, and throttling limits.
- **Advanced Query Logic**:
- [Throttling](ca://s?q=Notification_throttling) to enforce peruser limits.
- [Quiet Hours Filtering](ca://s?q=Quiet_hours_notification_filtering).
- [Deduplication](ca://s?q=Notification_deduplication) of similar alerts.
- [Aggregation](ca://s?q=Notification_aggregation) of lowpriority notifications.
- [PriorityBased Delivery](ca://s?q=Priority_based_notification_delivery).
- **Resilient Messaging**: Kafka decouples ingestion, processing, and delivery for fault tolerance.
- **Retry & Logging**: Failed deliveries retried with logging into MongoDB.
- **Analytics API**: `/analytics` endpoint for delivery stats, retries, and engagement metrics.
---
## 🎯 Objectives
- Ingest and validate notification requests via APIs.
- Process notifications in real-time or schedule them for future delivery.
- Respect **user preferences** (quiet hours, channel selection, throttling limits).
- Support **deduplication, aggregation, and prioritization** of alerts.
- Deliver notifications reliably with retry logic and logging.
- Provide analytics and monitoring endpoints for system visibility.
---
## ⚙️ Implementation Details
- **Backend Framework**: Node.js, Express.js
- **Messaging System**: Apache Kafka
- **Database**: MongoDB (notifications, preferences, analytics)
- **Job Scheduling**: node-schedule
- **ORM**: Mongoose
- **Containerization**: Docker + Docker Compose
---
## 🏗️ System Components
1. **Notification Ingestion & Validation Service**
- `/notify` API endpoint to receive requests.
- Validates payload and publishes to Kafka topic `notifications`.
2. **Notification Processing & Scheduling Engine**
- Real-time delivery for urgent/high-priority alerts.
- Scheduled notifications stored in MongoDB and checked periodically.
3. **User Preferences & Rules**
- Preferred channels (Email, SMS, Push).
- Quiet hours (Do Not Disturb).
- Notification limits (e.g., max 3/hour).
4. **Query Logic**
- **Throttling**: Restrict notifications per user-defined limits.
- **Quiet Hours Filtering**: Reschedule notifications outside quiet hours.
- **Deduplication**: Suppress duplicate alerts within a time window.
- **Aggregation**: Batch low-priority notifications into summaries.
- **Urgent Alerts**: Immediate bypass of scheduling.
5. **Notification Delivery Service**
- Channels: Email, SMS, Push (mock integrations).
- Retry mechanism for failed deliveries.
- Logging of statuses in MongoDB.
6. **Analytics & Monitoring**
- `/analytics` endpoint for delivery stats, retries, and engagement metrics.
---
## 🛠️ Installation & Setup
```bash
# Clone the repository
git clone https://github.com/akshay931/distri_notification_and_alert_system.git
cd distri_notification_and_alert_system
# Install dependencies
npm install
# Start services with Docker Compose
docker-compose up -d
```
## Installation
1. Clone the repository:
```bash
git clone https://github.com/akshay931/videoSDK.git
+3 -4
View File
@@ -652,10 +652,9 @@
}
},
"node_modules/mongoose": {
"version": "8.9.3",
"resolved": "https://registry.npmjs.org/mongoose/-/mongoose-8.9.3.tgz",
"integrity": "sha512-G50GNPdMqhoiRAJ/24GYAzg13yxXDD3FOOFeYiFwtHmHpAJem3hxbYIxAhLJGWbYEiUZL0qFMu2LXYkgGAmo+Q==",
"license": "MIT",
"version": "8.9.5",
"resolved": "https://registry.npmjs.org/mongoose/-/mongoose-8.9.5.tgz",
"integrity": "sha512-SPhOrgBm0nKV3b+IIHGqpUTOmgVL5Z3OO9AwkFEmvOZznXTvplbomstCnPOGAyungtRXE5pJTgKpKcZTdjeESg==",
"dependencies": {
"bson": "^6.10.1",
"kareem": "2.6.3",