At Pagoda Inc., we designed and deployed a recommender system to enhance user engagement on the NEAR social platform. By leveraging advanced machine learning for similarity detection, network analysis, and blockchain integration, we optimized user interactions and decentralized social connectivity.
Company: Pagoda Inc.
Role: Data Scientist
Network Dynamics and Community Structure
A core visualization from March 2023 provides insight into the structural dynamics of NEAR's social ecosystem, revealing distinct patterns of user engagement:
- Tightly connected influencer hubs: Key profiles such as
root.nearandmob.nearacted as central nodes, fostering organic community integration. - Isolated growth patterns: Profiles like
AuroraEcosystemNews.neardemonstrated high follower counts but remained detached from the broader network, highlighting fragmentation risks in decentralized platforms.
This analysis underscores how network topology influences decentralized social engagement, offering a framework for optimizing recommender system performance.
Near.social - Trending Users subset, March 2023.
Recommender System Development
The system was developed using a multi-layered approach, incorporating trending user detection, similarity analysis, and network optimization:
- Trending User Metrics: A custom ranking algorithm aggregated profile engagement statistics to identify trending users, significantly increasing visibility and interaction. This leaderboard was utilized in events and governance-related applications.
- Similarity Detection System: Multilingual embeddings and cosine similarity were deployed to detect thematic and contextual relationships between user profiles and content, refining recommendation precision.
- Network Enhancement via the HITS Algorithm: A Friends-of-Friends model was introduced, leveraging Hypertext Induced Topic Search (HITS) for second-degree connection discovery, fostering community expansion.
- Blockchain Integration for Privacy-Preserving Tracking: The system incorporated anonymized engagement metrics into the Blockchain Operating System (BOS), aligning with Web3 principles by securing data ownership and transparency while mitigating tracking concerns.
Technical Challenges & Scalability
Transitioning from local machine learning models to distributed computing (PySpark) on Databricks introduced complexity, particularly in adapting PyTorch-based solutions. The process required significant model optimization and parallelization to maintain efficiency across a decentralized infrastructure.
Impact & Future Directions
The recommender system redefined personalized content delivery in decentralized environments, enhancing user retention and engagement. Through robust data science methodologies, it bridged the gap between blockchain, AI-driven recommendations, and social networking.
Strategic Shifts & Reflections
By March 2024, the NEAR social platform and BOS were discontinued as Pagoda Inc. pivoted toward AI agent development, led by Ilya Polosukhin. This shift marked a departure from decentralized social networking, underscoring the challenges of sustaining Web3 ecosystems in evolving technological landscapes.
While the recommender system was retired, the insights gained remain applicable to future AI-driven social architectures, highlighting the interplay between decentralization, recommendation systems, and blockchain integration. The project serves as a case study in how technological ecosystems adapt, evolve, and redefine innovation trajectories.
