Parallel Multi-Layer Social Network Analyzer

The Parallel Multi-Layer Social Network Analyzer is designed to provide a scalable, high-performance solution for large-scale network analysis. By moving beyond traditional single-threaded computation—which suffers from massive wall-clock latencies and CPU core underutilization—this framework implements a Hybrid Parallel High-Performance Computing (HPC) architecture to execute context-specific graph mining algorithms on dense network topologies.

---

🛠️ Technologies & Frameworks (The HPC Stack)

To safely bypass the Python Global Interpreter Lock (GIL) and maximize multi-threading capabilities:

• Python 3.10 – Rapid development and rich graph theory primitives.
• mpi4py (Message Passing Interface) – Low-overhead inter-process messaging for distributed I/O, barrier synchronization, and collective communication (allgather).
• NetworkX – Multiplex graph construction, structural edge aggregation, and baseline linear comparisons.
• MS-MPI v10 Bindings – System-level MPI core engine.
• NumPy & Pandas – High-performance array operations and tabular logging of benchmarks.

---

📌 Main Functionalities

1. Multilayer Graph Construction & Transformation

• Programmatic Slicing: Converts flat edge lists into a 10-layer Simulated Multi-Layer Network (MLSN) using sequential node boundaries.
• Intra-Layer Isolation: Segregates "Within-Category" interactions into independent layer files (layer_0.txt → layer_9.txt).
• Inter-Layer Bridging: Routes cross-layer edges into a specialized ledger (inter_layer_edges.txt) to preserve high-dimensional multiplex structure.

2. Parallel Performance Optimization

• Distributed I/O Partitioning: Uses Modulo Round-Robin chunk assignment to reduce disk and memory scanning strain:
  $$\text{Assigned Chunks} = { i \in [0, 9] \mid i \pmod P == \text{Rank} }$$
• Topological Aggregation: Consolidates 925,872 raw edges into 357,322 unique edges using comm.allgather().

3. Asynchronous Functional Task Mapping

• Executes different algorithms concurrently on dedicated physical cores (-n 3):
  • Core 0 (Rank 0 - Master): Power-Iteration PageRank ($\alpha = 0.85$)
  • Core 1 (Rank 1 - Worker): Louvain Community Detection (modularity heuristic)
  • Core 2 (Rank 2 - Worker): Network density via Clustering Coefficients

---

📊 Verified Hardware & Performance Benchmarks

Experimental Environment

• CPU: Intel Xeon E-2176M @ 2.70GHz (6 cores, 12 threads)
• Memory: 32 GB High-Bandwidth RAM
• Target Database: SNAP Amazon Graph – 334,863 Nodes | 925,872 Raw Edges

Performance Metrics Comparison

Performance Metrics Evaluated	Sequential Baseline (1 Core)	Hybrid Parallel Framework (3 Cores)
Data Partitioning Style	Sequential FIFO Loop	Distributed Modulo Slicing
Algorithmic Execution Model	Linear Sequence (Wait-and-Run)	Asynchronous Task Mapping
Total Wall-Clock Runtime	69.7560 Seconds	34.5800 Seconds
Achieved Speedup Ratio ($S_p$)	1.00× (Baseline)	2.02×
Hardware Core Efficiency ($E_p$)	100.0%	67.3%
Net Processing Window Saved	Baseline	35.17 Seconds (Saved)

---

📈 Core Insights & Scalability

• Amdahl’s Law Validation: Direct MPI functional task mapping reduced runtime by 50.4%.
• Efficiency Analysis: Deviation from ideal 3× scaling to 2.02× due to inter-process communication overhead (pickling during allgather).
• Bottleneck Isolation: Louvain Community Detection (80.3% of sequential runtime) was effectively parallelized, reducing total wall-clock runtime to 34.58 seconds.

---