TraceProbe

GitHub Link: https://github.com/Reeti05Agarwal/TraceProbe

📖 About The Project

In digital forensics, investigators often face the monumental task of sifting through millions of Internet Protocol Detail Record (IPDR) logs to find evidence. This process is manual, time-consuming, and prone to error.

IPDR FlowAnalyzer was built to solve this problem. It is an intelligent, real-time data processing pipeline that ingests raw IPDR logs and automatically maps communication flows between an initiator (A-Party) and a recipient (B-Party). By identifying connections and flagging anomalies in real-time, our tool provides law enforcement with actionable intelligence, significantly reducing investigation time and increasing the efficiency of digital forensic support by over 30%.

Key Features

• Log Ingestion & Normalization: Ingests high-volume IPDR streams using Apache Kafka without data loss and Supports multiple formats (CSV, JSON, XML).
• A-Party to B-Party Mapping: Intelligently parses complex logs to identify and map initiator vs. recipient communications.
• Automated Anomaly Detection: Utilizes Apache Flink for complex event processing to flag suspicious patterns, such as connections to known malicious IPs, unusual data transfer volumes, DNS record frequency counts, Fake ID detection (geo mismatch, entropy analysis).
• Interactive Visualization: A user-friendly dashboard featuring:
  • Session timelines
  • Protocol Analysis
  • Geographical Map View: To visualize the physical locations of communicating parties.
  • Graph-Based Network View: To explore relationships and connections between different entities.
• Advanced Query & Search: A powerful search interface for investigators to filter and query specific sessions, IPs, or timeframes.
• Data Normalization: Handles various IPDR formats by normalizing them into a unified schema for consistent analysis.
• Security & Compliance:
  • Role-Based Access Control (RBAC)
  • TLS encryption for secure communication
  • Index-level security & audit logging

---

🏛️ Architecture

The system is designed as a distributed, scalable pipeline to handle massive data loads. 

---

🛠️ Tech Stack

This project leverages a modern, high-performance technology stack for real-time data processing and analysis.

Component	Technology
Data Streaming
Stream Processing
Backend & Logic
Frontend
Database	/
Containerization

---

🚀 Getting Started

To get a local copy up and running, follow these simple steps.

Prerequisites

• Docker and Docker Compose
• Python 3.9+
• Node.js and npm (for the frontend)

Installation & Setup

1. Clone the repository:

   git clone [https://github.com/your_username/ipdr-flowanalyzer.git](https://github.com/your_username/ipdr-flowanalyzer.git)
   cd ipdr-flowanalyzer

2. Set up environment variables:

   • Create a .env file from the env.example.
   • Fill in the necessary configuration details.

3. Launch services with Docker Compose:

   • This will start Kafka, Flink, and the required databases.

   docker-compose up -d

4. Set up the Backend:

   cd backend
   pip install -r requirements.txt
   python app.py

5. Set up the Frontend:

   cd frontend
   npm install
   npm start

---

📈 Usage

Once all services are running:

1. Open the application: Navigate to http://localhost:3000 in your browser.

2. Feed data into Kafka: Use the provided Python scripts in the /scripts directory to simulate an IPDR log stream.

   python scripts/produce_logs.py --file data/sample_ipdr.csv

3. View the Dashboard: Watch as the dashboard populates with real-time connections, and see anomalies being flagged as they are detected by the Flink job.