Iroh and iroh-gossip workshop for 39c3

Slides

Iroh doctor

iroh-doctor can be used to check connection status and performance. Install with cargo install iroh-doctor.

Chat

<iroh.computer/discord> channel #workshop

Wifi

You can use any wifi. You will always be able to establish a connection as long as you are connected to the internet in some way.

Prerequisites

This is a rust workshop, so you will need a rust toolchain installed. You can download it at https://rustup.rs/

If you like IDEs, rust-analyzer is a helpful tool. E.g. available as a plugin to vscode.

If you are new to rust, don't worry. The workshop is structured into multiple self-contained examples that can be run without any rust knowledge, and the code is easy enough to understand.

The workshop will introduce the basics of iroh using an echo example over raw QUIC streams. It will then use the gossip protocol to build a very simple chat application.

Intro

• Who wrote a simple TCP server before?
• Who has experience with rust?
• Who has cargo installed?

Disclaimer

The 0.9x releases are canary releases, things break more eagerly. The main branch is not compatible currently, it uses holepunching over QUIC Multipath. We'll release this soon, and 1.0 a little later.

On endpoint Ids

For a server, we want a stable endpoint ID. For a client, we don't care, it can be random.

An endpoint ID is an ed25519 keypair, which can be generated from just 32 bytes of random data.

To keep the endpoint id stable, we need to persist this private key somewhere. We don't want to deal with persistence in the examples, so we use an environment variable.

For all following examples, run them with IROH_SECRET=... to get a stable endpoint id.

if let Ok(secret) = env::var("IROH_SECRET") {
    // Parse the secret key from string
    SecretKey::from_str(&secret).context("Invalid secret key format")
} else {
    // Generate a new random key
    let secret_key = SecretKey::generate(&mut thread_rng());
    println!("Generated new secret key: {}", secret_key);
    println!("To reuse this key, set the IROH_SECRET environment variable to this value");
    Ok(secret_key)
}

Exercises

The workshop is structured into multiple exercises. Each exercise is a self-contained example and can be run using cargon run -p .

Echo 1

A simple echo service, to show the basics of iroh connections.

cargo run -p echo1

Connect side

We want to write a simple echo protocol that answers what it was sent.

Creating an endpoint to connect to the echo service:

let endpoint = Endpoint::builder()
    .bind()
    .await?;

Connecting:

const ECHO_ALPN: &[u8] = b"ECHO";
let connection = endpoint.connect(addr, ALPN).await?

Opening a stream:

let (send_stream, send_stream) = connection.open_bi().await?;

Writing the message:

send_stream.write_all(message.as_bytes()).await?;
send_stream.finish()?;

Reading the response:

let res = recv_stream.read_to_end(1024).await?;

Closing the connection:

conn.close(0u8.into(), b"done");
conn.closed().await;

Accept side

Accept a connection, two stage process.

let incoming = ep.accept().await?;
let conn = incoming.await?;

Accept a bidi(rectional) stream:

let (mut send_stream, mut recv_stream) = conn.accept_bi().await?;

Read the message and echo it back:

let msg = recv_stream.read_to_end(1024).await?;
send_stream.write_all(&msg).await?;
send_stream.finish()?;

Wait for the connection to close:

conn.closed().await;

Echo 2

Echo service from before, but done as a iroh protocol handler.

cargo run -p echo2

echo1 accepts exactly 1 connection, but then shuts down.

We don't want that. We want it to stay open.

We could just write an accept loop. But iroh has a functionality for this: Define a protocol handler

impl ProtocolHandler for EchoProtocol {
    async fn accept(
        &self,
        conn: Connection,
    ) -> AcceptResult<()> {
        ...
    }
}

Create a router and add the protocol handler

let router = Router::builder(ep)
    .accept(echo::ECHO_ALPN, echo::EchoProtocol)
    .spawn()
    .await?;

Wait for control-c, the router is handling accepts in the background:

tokio::signal::ctrl_c().await?;

Echo 3

Echo service from before, but show intricacies of node discovery.

cargo run -p echo3

These tickets are huge. Also, they contain information that might change. E.g. the current ip addr or relay URL.

Can we dial just with the node id, which does not change?

Accept

Configure discovery publishing

let ep = Endpoint::builder()
    .alpns(vec![echo::ECHO_ALPN.to_vec()])
    .secret_key(secret_key)
    .discovery_n0()
    .discovery_dht()
    .bind()
    .await?;

Now we can also use the short ticket that just contains the node id.

let ticket_short = NodeTicket::from(NodeAddr::from(addr.node_id));

The example will print some info on how you can see what is going on with discovery:

Looking up the published info on the DNS system:

dig TXT @dns.iroh.link _iroh.unt5ncmjw3g1ui7hfkdzqf6cdgxam446i4apsseghkksg1jc3g7o.dns.iroh.link

Looking up the published info on the bittorrent mainline DHT

https://app.pkarr.org/?pk=unt5ncmjw3g1ui7hfkdzqf6cdgxam446i4apsseghkksg1jc3g7o

Connect

Configure discovery resolution.

The connect side will generate a random node id that nobdoy cares about. We don't want to publish our node id.

let ep = Endpoint::builder()
    .add_discovery(PkarrResolver::n0_dns())
    .add_discovery(DhtDiscovery::builder().build().unwrap())
    .bind()
    .await?;

Gossip 1

One broadcaster & joiner.

cargo run -p gossip1

Broadcaster

The gossip protocol is a struct which implements the ProtocolHandler trait. This can be passed into the Router to accept connections.

It is however a more complex protocol that also needs to initiate connections, so building it requires the endpoint as well. It is started using a spawn method since it runs several background tasks managing all the connections to the gossip neighbours.

let gossip = Gossip::builder().spawn(ep.clone());
let router = Router::builder(ep.clone())
    .accept(iroh_gossip::ALPN, gossip.clone())
    .spawn();

Gossiping is done on a "topic". Topics are identified by a TopicId which is usually some random 32-bytes. In these examples however we use an easy-to create TopicId containing all zero-bytes. Don't do this in production!

let topic_id = TopicId::from_bytes([0u8; 32]);

And finally all topics must be joined. We join the topic, then wait until another node joins. Then we broadcast a message before exiting:

let mut topic = gossip.subscribe(topic_id, vec![]).await?;
topic.joined().await;
topic.broadcast("welcome".into()).await?;

Joiner

On the joining side we want to subscribe to the topic and wait for messages. We use subscribe_and_join so that we wait until we have seen at least one other peer: our broadcaster.

let mut topic = gossip
    .subscribe_and_join(topic_id, vec![bootstrap_id])
    .await?;

Notice that we also supplied the bootstrap_id: we need to supply the subscriber with the EndpointId of the broadcaster. In a larger gossip network we can use multiple so-called "bootstrap nodes".

Gossip relies on dial-by-id, so we supply an EndpointId instead of a full EndpointAddr. Interally however it also has its own discovery service so that it uses addressing information from neighbours it discovers.

Let's receive some events:

    while let Some(event) = topic.try_next().await? {
        match event {
            Event::NeighborUp(endpoint_id) => todo!(),
            Event::NeighborDown(endpoint_id) => todo!(),
            Event::Received(message) => todo!(),
            Event::Lagged => todo!(),
        }
    }

These are all the events that you can receive from gossip:

• neighours joining and leaving
• messages
• a warning if you don't keep up with the received events

Gossip 2

Multiple messages, avoiding deduplication of messages.

cargo run -p gossip2

When gossiping messages it is possible that loops occur: the same message could appear multiple times. To stop messages like these from existing forever duplicate messages are ignored.

These 3 messages being broadcase will only result in two messages being received:

    topic.broadcast("welcome".into()).await?;
    topic.broadcast("welcome".into()).await?;
    topic.broadcast("welcome2".into()).await?;

The solution is to make sure all messages sent are unique.

We take two measures:

• Each sender has a counter value it adds to messages, making sure to never re-use the value: a nonce.
• Each sender signs their messages. Thus even identical messages from different senders will be different.

This is implemented in the SignedMessage struct:

SignedMessage::new(secret_key: &SecretKey, contents: String) -> Self

Creates a new unique message, incrementing the nonce. The signature is made from the nonce and the contents:

let nonce = NONCE.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
let signed_payload = postcard::to_stdvec(&(nonce, &contents)).expect("always succeeds");
let signature = secret_key.sign(&signed_payload);

Here postcard is used as a simple but powerful serialisation library. Postcard is again used to encode the entire message into bytes that can be broadcast again.

Gossip 3

A single node, split broadcaster and listener.

cargo run -p gossip3

The gossip topic can be split into a gossip sender and receiver:

let (sender, receiver) = topic.split();

This is convenient as it is often easier to write dedicated tokio tasks rather than having to write one task that has to handle events to broadcast messages as well as has to receive events.

let send_task = tokio::task::spawn(broadcast(secret_key, sender, message));
let recv_task = tokio::task::spawn(listener(receiver));

The tasks run their respective loops forever. But now as part of a single node that can both send and receive messages. The example no longer distinguishes between a broadcast sender and receiver, both run exactly the same code and are equals in the network.

Finally we add nice error handling and shutdown again by awaiting for failing tasks or the Ctrl-C signal using the FutureExt::race trait method.

Gossip 4

Use interactive messages (no iroh changes here, just rust code).

Publish to a content discovery service, and use that service to find providers for the content on the receive side

cargo run -p sendme4

To read messages from the terminal we need to read standard input:

let mut buffer = String::new();
let stdin = std::io::stdin();
println!("> type a message and hit enter to broadcast...");
stdin.read_line(&mut buffer)?;

Now we want to broadcast each line of input. So we need to create a new unique message:

        let msg = SignedMessage::new(&secret_key, text);
        sender.broadcast(msg.encode()).await?;

Due to a quirk from tokio however, the reading of lines has to run in a thread rather than a tokio task. We can communicate between the two using a channel:

let (line_tx, mut line_rx) = tokio::sync::mpsc::channel(1);
let thread = std::thread::spawn(move || input_loop(line_tx));
while let Some(text) = line_rx.recv().await {
    // create SignedMessage & broadcast it
}

Notes

Note: the workshop is using an alpha version of iroh.

The stable version of iroh 0.35. The 0.9x series is changing rapidly as we work towards a 1.0 release.

Docs are at https://docs.rs/iroh-blobs/latest/iroh/