gnutella protocol

I've started work on a program which utilizes the gnutella protocol (as of version 0.48). Basically, you connect a SOCK_STREAM (tcp) socket to any other gnutella server, send a GNUTELLA CONNECT/0.4[lf][lf] and expect back a GNUTELLA OK[lf][lf]. At this point the server expects you to identify yourself. You send a type 0x00 message to whoever you just connected to, and the server responds with how many files it is sharing, and the total size of those files (in KB). You'll also get a response from everybody connected to the machine you connect to, and so on, until the TTL expires on the message.

At this point, the server will start bombarding you with information about other servers (which fills the host catcher, and gnutellanet stats). You'll also get search requests. You're supposed to decrement the TTL and pass it on to any other servers you're connected to (if TTL > 0) If you have no matching files you can simply discard the packet, otherwise you should build a query response to that message and send it back from where it came.

The header is fixed for all message and ends with the size of the data area which follows. The header contains a Microsoft GUID (Globally Unique Identifier for you nonWinblows people) which is the message identifer. My crystal ball reports that "the GUIDs only have to be unique on the client", which means that you can really put anything here, as long as you keep track of it (a client won't respond to you if it sees the same message id again). If you're responding to a message, be sure you haven't seen the message id (from that host) before, copy their message ID into your response and send it on it's way. That message ID is followed by a function ID (one byte), which looks to be a bitmask. The function ID indicates what type to do with the packet (search request, search response, server info, etc). The next field is a byte TTL. Every packet you recieve you should dec (or -- for the C guys) the TTL and pass the packet on if the TTL is still > 0 (i.e. if (--hdr.TTL) { [pass on] }, god I love C). You should also inc the hop count. Seems redundant? Well, some people have smaller TTLs, and you have the right to drop any message you want to based on its hop count. The header finishes up by telling us how large the function-dependant data that follows is.

Searches: Easy, just build a type 0x80 packet, add a WORD for the minimum connection speed (in kbps), then the null terminated string. There isn't a response from people who have no match, but a result will come back as a type 0x81 message. There will be a gnutella_query_response_hdr followed by N gnutella_query_response_rec_hdr and double NULL terminated filenames. To finish this up, there's a gnutella_query_response_ftr with the full 128 bit (16 byte) client ID of the server that found the result.

Downloads and Uploads:These are a POC. If you want a file from a server, you connect to the server, and send an HTTP request for it. The URL is of the form /get/[file_id]/[filename]. The file id was returned with the search result. The gnutella HTTP server also supports resuming a transfer via the Content-range: HTTP header. If you're just curious, the User-Agent is gnutella. You can actually load up Netscape, and get a file from a Gutella server. Pretty cool, eh? Here's a dump of what a HTTP request looks like:

Ok, you all are wondering, sure the verbal theory is great, but why not just give us the code! This is my source code: Here's what I got so far. I know, it's in pascal, but what the heck, I'm a Delphi programer 18 hours a day (and a beer drinker the rest). This is a component you can add to your palette which handles connecting to gnutellanet simply drop on your form, link up some event handlers, and call ConnectNewHost to get started. BeginNewSearch() is also something you might be interested in.

So, what have I got running? I've got a client which will connect to multiple servers, host catch, do search requests (and get the results). Of course the search monitor works too. Here are some screenshots. It's written in Delphi, and the code is up above.

Here's a sample log file which shows the protocol in action.

Awlright, some people have read this document and sill keep asking me things like "What's in the first 10 bytes of the header?", so for people who can't figure out how what the syntax of a Delphi record looks like, or can't read english so well, here's some nice tables:

gnutella_header

Function IDs

gnutella_ping_response

gnutella_query_hdr

gnutella_query_response_hdr

gnutella_query_response_rec

gnutella_push_req

Routing

An issue everyone wants to ask me about nowadays is routing. "Do I forward every packet I see to every connected host?" Holy Jesus no! That would swamp the network with duplicate packets (which it already is). Here's the secret. For simplicity sake, TTL is not discussed in this section

(Forgive the non-straight lines, but the internet's like that)

Imagine yourself as node 1 in the above diagram. You have direct gnutellanet (physical socket) connections to nodes 2, 3, 4, and 5. You have reachable hosts at nodes 6 thru 13.

1. You get an ping message (function 0x00) from 2 with a message id of x.
2. Lookup in your message routing table [message x, socket ???]
3. Not there? Save [message x, socket 2] in the list.
4. Respond with an Ping Response (0x01), message id x to node 2.
5. Send the function 0x00 message to nodes 3, 4, and 5 (not 2!!).
6. Node 3 will respond with Ping Response (0x01), message id x.
7. Forward the message to whoever in the list has [message x, socket ???], since this packet is being routed and not broadcast, there is no need to check for if it is a duplicate, as routed messages don't make loops.
8. Do the same thing with responses from 4 and 5.
9. Since 3 thru 5 will also pass the message on to 8 thru 13, you'll also get a 0x01 from them too.
10. Problem:Node 3 is connected to 10 who is connected to 4 who is connected to you! It's OK! You lookup in your route list [message x, socket ???]... It's already there! You drop the message, do not respond to 4, do not forward to anyone!

Here's the basic mechanics, described in the example above:

• If the low bit of the function ID (f) is 0, look for [message x, socket ???]. If it's already there, drop the message. If it isn't, add it as [message x, socket s], respond to socket s, and forward to all connected clients except socket s (the one you got the message from).
• If the low bit of the function ID (f) is 1, look for the socket which matches [message x, socket ???] and forward the message to that connection only.
• If the low bit if the function (f) is not 0 or 1, you need to stop letting an inifite number of monkeys use your machine while they work on their Hamlet script.

TTL and Hops

"How many computers the packet can go through before it will stop being passed around like a whore" - Nouser (#gnutella on efnet)