> ## Content Index > Fetch the complete content index at: https://scopal-affairs.com/llms.txt > Use this file to discover other available public pages before exploring further. # Signal Meets Structure - URL: https://scopal-affairs.com/software/social-connectivity-nears-recommender-system/ - Published: 2023-10-30T17:54:00.000Z - Updated: 2026-08-20T07:21:05.000Z - Author: Daniel - Tags: software, data-science, #beleg ## Recommender Systems in a Fragmented Decentralized Social Graph *Network topology, weak signals, and the limits of social recommendation* --- ### Abstract This article describes the design and deployment of a recommender system for the NEAR social platform at Pagoda Inc.. The system combined similarity detection, network analysis, and blockchain-constrained data access to improve discoverability and engagement in a decentralized social graph. Rather than focusing on model performance alone, this work revealed how **network topology fundamentally constrains recommendation outcomes**. Although the system was later retired alongside near.social, the insights generalize to decentralized social architectures. --- ### 1\. System Context The NEAR social platform represented a decentralized social graph with: - open profile creation - weak identity guarantees - minimal enforced structure - highly uneven engagement patterns The recommender system was designed not to create communities, but to **amplify meaningful connections within an already fragmented topology**. --- ### 2\. Observing the Network A directed graph snapshot from March 2023 revealed clear structural patterns: - **Dense influencer hubs** Profiles such as `root.near` and `mob.near` acted as central connectors, enabling organic community integration. - **Isolated high-follower nodes** Profiles like `AuroraEcosystemNews.near` exhibited large follower counts but weak second-degree connectivity, resulting in brittle visibility spikes rather than durable embedding. This highlighted a core insight: > Follower count alone is a poor proxy for social integration. --- **Near.social - Trending Users subset, March 2023.* --- ### 3\. What a Recommender Can — and Cannot — Do These observations reframed the problem. In decentralized social graphs: - recommendation systems do not create structure - they amplify existing topology A recommender can surface connections, but it cannot compensate for missing second-degree connectivity. --- ### 4\. System Architecture The recommender followed a layered design: #### Trending User Detection A custom ranking aggregated engagement signals to identify trending profiles. The resulting leaderboard was used in governance and event-related contexts. #### Similarity Detection Multilingual embeddings and cosine similarity detected thematic overlap between profiles and content, enabling contextual recommendations beyond follower graphs. #### Network Expansion via HITS A Friends-of-Friends model leveraged the HITS algorithm to surface second-degree connections, emphasizing the distinction between hubs and authorities. HITS was chosen over PageRank to preserve role differentiation, though it proved sensitive to sparse local connectivity. --- ### 5\. Blockchain as Constraint, Not Optimization Engagement signals were integrated into the BOS with anonymization and limited granularity. The blockchain layer primarily acted as a **constraint**: - limiting tracking resolution - enforcing data ownership boundaries - preventing centralized behavioral profiling It did not optimize recommendation quality, but shaped what was permissible. --- ### 6\. Engineering Challenges Scaling from local experimentation to distributed computation (PySpark on Databricks) introduced non-trivial complexity: - adapting PyTorch-based workflows - managing parallelism over sparse graphs - balancing compute cost against marginal recommendation gains These challenges reinforced a key lesson: in decentralized systems, **infrastructure complexity grows faster than model sophistication**. --- ### 7\. Impact and Limits The system improved short-term discoverability and engagement. However, long-term structural effects could not be empirically validated. This exposed a recurring challenge in decentralized ecosystems: > Interventions are easy to deploy, but difficult to observe longitudinally. --- ### 8\. Strategic Shift and Retrospective By March 2024, after a huge relaunch in August 2023, near.social and BOS were discontinued as Pagoda Inc. pivoted toward AI agent development, led by Ilya Polosukhin. The recommender system was retired. The insights remain. --- ### 9\. Core Takeaway > In decentralized social graphs, recommender systems do not create communities. > They amplify existing structure. The quality of recommendations is bounded less by model choice than by network topology. --- *Field report. Partial data. Durable insight.* --- ### **Additional Resources** - [\[NEAR Recommender Prototype on GitHub\]](https://github.com/scopalaffairs/near%5Frecommender?ref=scopal-affairs.com) - [\[NEAR Recommender - Trending on the BOS (Public Access)\]](https://dev.near.org/near/widget/PeoplePage?tab=trending&ref=scopal-affairs.com) - [\[NEAR Recommender - Recommended on the BOS (NEAR Profile Required)\]](https://dev.near.org/near/widget/PeoplePage?tab=recommended&ref=scopal-affairs.com)