The purpose of this extension (Extension for Peers to Send Metadata Files) is to allow clients to join a swarm without having to download a torrent file (.torrent) first. Instead, this extension allows clients to download metadata from similar nodes. It enables support for magnet links, which contain enough information on a webpage to join the swarm (i.e., info hash). In simple terms, this extension allows users to download the required files without downloading the torrent file (.torrent).
Metadata#
In this extension, only the metadata portion, called the "info-dictionary," is transmitted. This data is sufficient for other clients to know how to join a specific swarm and start downloading file data. By using the info-hash to verify the integrity of the metadata, clients can ensure that they download metadata from reliable sources without being affected by malicious modifications.
The metadata is divided into blocks of size 16KiB (16384 bytes) for processing. These metadata blocks are indexed starting from 0. Each block has a size of 16KiB, except for the last block, which may be smaller than 16KiB.
Extension Handshake#
When a client establishes a connection with other nodes, they communicate through handshake messages. The metadata extension uses the extension protocol specified by the Extension Protocol to announce its presence to other nodes. Specifically, it adds the "ut_metadata" entry to the "m" dictionary of the extension handshake message to identify the message code used. Additionally, it adds the "metadata_size" field (not in the "m" dictionary) to specify the number of bytes of metadata. In other words, when a client establishes a connection and performs an extension handshake with other nodes, if the metadata extension is available, the client will send an extension handshake message containing the "ut_metadata" and "metadata_size" fields to inform other nodes that it supports obtaining metadata.
Example extension handshake message:
{'m': {'ut_metadata', 3}, 'metadata_size': 31235}
Extension Messages#
Extension messages are encoded. There are three different types of messages:
- Request
- Data
- Reject
Request Message#
A request message is a dictionary without additional keys (i.e., additional information). If a node supports this extension, it will respond with a reject or data message, and the response piece must be the same as the one requested. Specifically, both the request and response involve the same data block (piece).
In this network, a node must verify that any data block (piece) it sends has passed the info-hash check. In other words, a node cannot run the SHA-1 algorithm to verify that the generated hash matches the info-hash until it has obtained the complete metadata. If a node does not have complete metadata, it must send a "reject" message to reject the request when it receives a metadata request from another node. This is because only nodes with complete metadata can perform the info-hash check correctly. Thus, the network ensures the integrity and accuracy of file transfers, preventing potential malicious operations or data corruption.
Example:
{'msg_type': 0, 'piece': 0}
d8:msg_typei0e5:piecei0ee
The request message is requesting the first metadata piece.
Data Message#
The data message adds a new entry, "total_size," to the dictionary. This key has the same semantics as the "metadata_size" in the extension header and its value is an integer. This key represents the total size or length of the entire data block and indicates how much data the receiver needs to receive to fully reconstruct the original data.
When transmitting metadata, the dictionary needs to be serialized into binary data and prefixed with a length according to the protocol requirements. This length prefix should include not only the length of the dictionary itself but also some additional information about the metadata, called metadata piece. The metadata piece is not part of the dictionary but part of the message. If the metadata piece is the last part of the metadata, its size can be smaller than 16KB. However, if it is not the last part, its size must be 16KB.
Example:
{'msg_type': 1, 'piece': 0, 'total_size': 3425}
d8:msg_typei1e5:piecei0e10:total_sizei34256eexxxxxxxx...
x represents binary data (metadata).
Reject Message#
A reject message does not contain any additional keys or values. If received, it should be understood that the corresponding peer does not have the requested metadata. This means that the peer cannot provide the required data fragments. Clients can protect against flood attacks by starting to reject them after serving a certain number of request messages. This number is usually the requested number of metadata blocks multiplied by a factor.
Example:
{'msg_type': 2, 'piece': 0}
d8:msg_typei1e5:piecei0ee
Magnet URI Format#
The magnet URI format is as follows:
v1: magnet:?xt=urn:btih:<info-hash>&dn=<name>&tr=<tracker-url>&x.pe=<peer-address>
v2: magnet:?xt=urn:btmh:<tagged-info-hash>&dn=<name>&tr=<tracker-url>&x.pe=<peer-address>
<info-hash>
<tagged-info-hash>
<peer-address>
xt
dn
tr
x.pe
For more details, refer to this article.