Specification IPFS - Bittorrent

[vk496]

IPFS - Bittorrent v2 Gateway Specification Proposal

Abstract

This proposal outlines a specification for an open-source IPFS-Bittorrent v2 gateway that enables seamless interoperability between the InterPlanetary File System (IPFS) and Bittorrent v2 protocols. The gateway aims to bridge the decentralized storage and content addressing capabilities of IPFS with the efficient peer-to-peer file distribution of Bittorrent v2.

Introduction

IPFS provides content-addressed, permanent storage with Merkle DAGs, while Bittorrent v2 offers robust, swarm-based file distribution with improved metadata handling. A gateway between these systems would allow users to leverage the strengths of both protocols, enabling cross-protocol content discovery, sharing, and retrieval.

Problem Statement

Currently, IPFS and Bittorrent v2 operate in separate silos:

• IPFS content is not accessible via Bittorrent clients
• Bittorrent swarms cannot benefit from IPFS's content addressing and deduplication
• Limited interoperability between decentralized storage and distribution networks

Goals and Objectives

1. A file located in IPFS could be discovered and downloaded from a Bittorrent V2 client
2. Users could easily share with IPFS content that is already being shared in torrent

Specific Objectives

• Define mapping between IPFS Content Identifiers (CIDs) and Bittorrent v2 info-hashes
• Specify gateway protocols for content discovery and retrieval
• Establish metadata exchange formats between the two systems
• Ensure backward compatibility with existing IPFS and Bittorrent implementations

State of the art

A torrent infohash is a SHA256 of a bencoded structure that defines the content of the torrent. A JSON representation of it can be seen with ./torrenttools info --raw sample_torrent_v2.torrent

{
    "meta version": 2,
    "name": "sample_torrent",
    "piece length": 262144,
    "file tree": {
        "hello_world.txt": {
            "": {
                "length": 12,
                "pieces root": "<merkle root 1894a19c85ba153acbf743ac4e43fc004c891604b26f8c69e1e83ea2afc7c48f>"
            }
        },
        "lorem_ipsum.txt": {
            "": {
                "length": 32768,
                "pieces root": "<merkle root 83de59143142fed75ee2965b30a0e00222b805c3556b2c663016005b53d1b59c>"
            }
        },
        "netboot.xyz.iso": {
            "": {
                "length": 2404352,
                "pieces root": "<merkle root 9baad3272ac05d9010f424b3506d027d32ec26d07642e784ab0fdeabb3c6719e>"
            }
        },
        "subfolder": {
            "dvdisaster-v0.79.10-pl5-x86_64.AppImage": {
                "": {
                    "length": 27949560,
                    "pieces root": "<merkle root 8e3f8857778140565d7e7407778c84d5600d60f153b7f09bf5bc686ed5810ed8>"
                }
            }
        }
    }
}

Every file in Bittorrent V2 is chunked as blocks of 16KiB except last one (no padding exists). A merkle root hash is calculated as binary tree of hashes:

More details can be found in https://blog.libtorrent.org/2020/09/bittorrent-v2/

The same file will always generate the same merkle tree. However, the bittorent swarm is linked to the infohash (torrent file) and not the merkle tree hash of the file. That means, same file in 2 different torrents will no be found between them.

On the other hand, IPFS every piece of data is a block referenced by a CID (a hash derivated from the content). Its a global descentralized network, so only 1 swarm exists. If you have the CID, you can get the content

Architecture

Bittorrent V1

Bittorrent V1 will be out of scope initially. I would focus on implementing V2 protocol, then hybrid and, maybe, the V1. The reason is that V1 hash everything sequentially, which makes very very difficult to design something compatible with IPFS. V2 is the best fit for IPFS

Node

There should be running a node/daemon that will join bittorrent DHT to announce its own infohash and answer to other peers as a valid peer. It will act as a "server torrent", only serving data

Torrent files

A user will have a folder where will be all .torrent files. This folder will be used as reference for announcing what torrents does the user have. The files must have a special chunking so the infohash is a unique chunk. Usually, that would mean that a .torrent file will have the following leafs layout:

• First 9 bytes (or more) chunk
• Second chunk, representing the infohash. It CID will match the infohash
• The remaining chunks (default 256KiB chunking)

Data inside IPFS

• Files inside IPFS must be UnixFS files, so they can be seen from the IPFS Desktop app
• They must have a static chunking of 16KiB, so they will match the blocksize of torrent
• They must use RAW leafs, so the CID (sha256) can be used directly for the calculation of the merkle tree hash

Bittorrent finding infohash

When then GW is asked for a infohash, it will directly request the infohash CID. I'm not sure if it should be data that only the user have or globally look for the CID (I would say globally)

Peer connecting to the GW

Since torrent DHT is for finding peers, the GW must announce himself. Maybe libp2p is a good fallback when the GW is not publicly reachabe?

Bittorrent peer initial setup

The handshake would announce that is a V2 torrent. Regarding to the bitfield, I think that the GW must announce that it have all pieces (even if is not true), or other peers will never request any data

Bittorrent data request

The tricky part. The peers will ask for blocks (16KiB except last one), specifying a index (torrent level).

Since files are deterministically chunked and we have all files and sizes of the torrent (info dictionary), we can know what exactly file and block is requested.

Knowing that, we can request to IPFS this specific file (we have its merkle tree root hash). I think the only possible solution to associate Merkle Root Hash and the UnixFS CID is having a custom DHT in IPFS. A key-value association between a merkle tree hash and a CID pointing to the file. A initial idea, would be that for a merkle tree root hash (sha256) request, the answer would be a type (default 0? Maybe in the future support more than UnixFS) and the CID pointing to the UnixFS file

Having the file CID, we can answer with the data, since it contains leafs of 16KiB already. If the CID is not reachable (timeout?), we would ask with a reject to the bittorrent peer who asked for that part

Merkle proof

A special request from a bittorrent peer would be the merkle proof. The unixFS DAG is a balanced tree, not a flat array. A efficient way to get the CIDs would be desirable, so we reduce the iteration over the tree

IPFS bt2 DHT

A custom DHT feels like the best idea, but it needs to take in account bad actors. Validation of the data are done in the merkle proof, but some basic mechanisms for discarting bad actors are very welcome

No data duplication

One of the things that MUST exist is a comfortable way to import your already existent folder of torrent data (assuming TBs) without duplicating. IPFS have a --nocopy argument that must be enforce by default, but its not clear for me how the user should add his folder the easies way possible (Patching IPFS Desktop? Custom GUI?). Initially it will be a CLI interface, but it should make easy to people share its content, so other IPFS/torrent users could potentially benefit from it

Others

• IPFS IPLD?
• Monitor changes?
• Add merkle tree information to every UnixFS file when importing/adding?

---

This is an initial proposal. All aspects are subject to community review and revision.

[HavenCTO]

The whole idea behind this IPFS-BitTorrent v2 gateway proposal sounds nice on paper, but it really ignores the massive friction that happens when you try to smash two completely different security models together. A much safer and more realistic approach is to just treat BitTorrent as a way to grab the data, then encrypt it and republish it on IPFS with actual access control, rather than pretending both networks are part of the same swarm.

The core problem is that BitTorrent and IPFS just don't trust the same things. BitTorrent is totally public, does best-effort replication with untrusted peers, and binds everything to an infohash. IPFS is all about content-addressed graph retrieval, but it has totally different discovery, pinning, and policy problems. If you try to bridge them at the protocol level using custom DHTs and aligned chunking, you end up inheriting neither ecosystem's security and you just add a brand new trust anchor that someone has to govern forever.

Then there's the issue of prematurely announcing a full bitfield when you can't actually serve arbitrary blocks. That completely misaligns with what peers expect, wastes connection slots, and degrades the whole swarm's health when IPFS lookups fail. It's way better to just seed honestly or use a dedicated cache with real availability.

And the idea that a torrent's data is magically going to match an IPFS CID just doesn't work in reality. For that to happen, the torrent has to be packaged using very specific IPFS-internal settings, like exactly how it's split into blocks and how folders are structured. Since almost no torrents are packaged with these exact settings in mind, the bridge only works for a tiny, custom-crafted subset of content instead of being a universal bridge.

A much more robust architecture is to just ingest, encrypt, and gate access. You download via BitTorrent normally without any premature bitfields, encrypt the payload before it becomes the canonical IPFS artifact, upload the ciphertext with minimal public metadata to IPFS, and enforce access control using verifiable credentials or on-chain state. The downside is that because this requires encryption and access control, users can't just use a standard BitTorrent client to download and open the files automatically. They need a specialized app that handles the decryption and authorization. But honestly, that's worth it, because an open system without any rules or oversight will inevitably be abused by bad actors, breaking the network for everyone.

[vk496]

Hey @HavenCTO , sorry for taking me that long to reply. I appreciate your feedback

The idea of this project is actually to solve all that gaps and be the gateway between both. The security concerns will be the weakest of both systems, but never something that we introduce (because of the content addressable system).

I didnt had time to document a lot, but a POC is tested and lay under https://github.com/vk496/ipfs2bt/tree/alpha

I was able to download a file with qBitorrent that was only existing in IPFS through the gateway