Recommender Systems in a Fragmented Decentralized Social Graph
Network topology, weak signals, and the limits of social recommendation
Abstract
This article describes the design and deployment of a recommender system for the NEAR social platform at Pagoda Inc..
The system combined similarity detection, network analysis, and blockchain-constrained data access to improve discoverability and engagement in a decentralized social graph.
Rather than focusing on model performance alone, this work revealed how network topology fundamentally constrains recommendation outcomes.
Although the system was later retired alongside near.social, the insights generalize to decentralized social architectures.
1. System Context
The NEAR social platform represented a decentralized social graph with:
- open profile creation
- weak identity guarantees
- minimal enforced structure
- highly uneven engagement patterns
The recommender system was designed not to create communities, but to amplify meaningful connections within an already fragmented topology.
2. Observing the Network
A directed graph snapshot from March 2023 revealed clear structural patterns:
- Dense influencer hubs
Profiles such asroot.nearandmob.nearacted as central connectors, enabling organic community integration. - Isolated high-follower nodes
Profiles likeAuroraEcosystemNews.nearexhibited large follower counts but weak second-degree connectivity, resulting in brittle visibility spikes rather than durable embedding.
This highlighted a core insight:
Follower count alone is a poor proxy for social integration.
Near.social - Trending Users subset, March 2023.
3. What a Recommender Can — and Cannot — Do
These observations reframed the problem.
In decentralized social graphs:
- recommendation systems do not create structure
- they amplify existing topology
A recommender can surface connections, but it cannot compensate for missing second-degree connectivity.
4. System Architecture
The recommender followed a layered design:
Trending User Detection
A custom ranking aggregated engagement signals to identify trending profiles.
The resulting leaderboard was used in governance and event-related contexts.
Similarity Detection
Multilingual embeddings and cosine similarity detected thematic overlap between profiles and content, enabling contextual recommendations beyond follower graphs.
Network Expansion via HITS
A Friends-of-Friends model leveraged the HITS algorithm to surface second-degree connections, emphasizing the distinction between hubs and authorities.
HITS was chosen over PageRank to preserve role differentiation, though it proved sensitive to sparse local connectivity.
5. Blockchain as Constraint, Not Optimization
Engagement signals were integrated into the BOS with anonymization and limited granularity.
The blockchain layer primarily acted as a constraint:
- limiting tracking resolution
- enforcing data ownership boundaries
- preventing centralized behavioral profiling
It did not optimize recommendation quality, but shaped what was permissible.
6. Engineering Challenges
Scaling from local experimentation to distributed computation (PySpark on Databricks) introduced non-trivial complexity:
- adapting PyTorch-based workflows
- managing parallelism over sparse graphs
- balancing compute cost against marginal recommendation gains
These challenges reinforced a key lesson:
in decentralized systems, infrastructure complexity grows faster than model sophistication.
7. Impact and Limits
The system improved short-term discoverability and engagement.
However, long-term structural effects could not be empirically validated.
This exposed a recurring challenge in decentralized ecosystems:
Interventions are easy to deploy, but difficult to observe longitudinally.
8. Strategic Shift and Retrospective
By March 2024, after a huge relaunch in August 2023, near.social and BOS were discontinued as Pagoda Inc. pivoted toward AI agent development, led by Ilya Polosukhin.
The recommender system was retired.
The insights remain.
9. Core Takeaway
In decentralized social graphs, recommender systems do not create communities.
They amplify existing structure.
The quality of recommendations is bounded less by model choice than by network topology.
Field report. Partial data. Durable insight.
