HackerPapers

arXiv:1009.3681cs.NI

Efficient Indexing of the BitTorrent Distributed Hash Table

Aaron Grunthal

Abstract

The following paper presents various methods and implementation techniques used to harvest metadata efficiently from a Kademlia Distributed Hashtable (DHT) as used in the BitTorrent P2P network to build an index of publicly available files in the BitTorrent ecosystem. The indexer design makes various tradeoffs between throughput and fairness towards other DHT nodes while also being scaleable in a distributed environment.

arXiv comments
Journal reference
DOI
Subjects
cs.NINetworking and Internet Architecture

On Hacker News

Linked by 2 identifiable commenters in 2 comments across 2 threads, from Mar 2013 to Dec 2014.

Comments linking this paper, per year.

Discussed in comments2 threads

  1. Crawling BitTorrent DHTs for Fun and Profit (2010) [pdf] 1 comment

    • Really cool idea. Using this paper and the referenced one at http://arxiv.org/pdf/1009.3681.pdf%3Forigin%3Dpublication_de... I was able to make a proof-of-concept passive DHT crawler in a few hours.

      rogerbraunView on HNParent context

  2. The Pirate Bay Is A Trailblazer In Technical Resilience 1 comment

    • Speaking of resilience, I wrote a paper [1] 3 years ago about how to rebuild torrent sites (like TPB) within a couple hours of a denial of service (legal or otherwise) using the Vuze client's DHT. There was some similar concurrent work [2] on the Mainline DHT by Aaron Grunthal.…

      _cbszView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.