Add libp2phttp command

[MarcoPolo]

This lets you make http requests over libp2p streams.

The strategy I went with was to listen on a unix socket and proxy any HTTP requests received on that socket to the target. This lets us reuse curl to do the actual HTTP parts.

Example:

$ vole libp2p http '/ip4/127.0.0.1/udp/4002/quic-v1/webtransport/certhash/uEiA14HNun3qVa6Z2XXinYHwx_XxGgTuCUJ_yWyzuU_AXcw/certhash/uEiBBv5HyzsoM_k-sxUOTm-VkDdNhWJpUyuk-7IHN2mm1EA/p2p/12D3KooWJzkorELQQRrWBwePjkTwkeJcLya9LZLLVMGHmRj62PZB'

Proxying on:
/var/folders/6t/nky73jj94hlcpcf1gf2mljb00000gn/T/libp2phttp-4159826572.sock

Example curl request:
curl --unix-socket /var/folders/6t/nky73jj94hlcpcf1gf2mljb00000gn/T/libp2phttp-4159826572.sock http://example.com/

In another terminal:

$  curl --unix-socket /var/folders/6t/nky73jj94hlcpcf1gf2mljb00000gn/T/libp2phttp-4159826572.sock http://example.com/

Hello World

[aschmahmann]

Thanks for adding this 🙏. This looks a lot like https://github.com/ipfs/kubo/blob/master/docs/experimental-features.md#p2p-http-proxy.

Do you think adding it's sister https://github.com/ipfs/kubo/blob/master/docs/experimental-features.md#ipfs-p2p (i.e. TCP proxying over libp2p streams) would be valuable as well (would be a separate PR)?

[aschmahmann]

Could this logic get into trouble if say /ws/tls is being used for a relay and combined with /http?

Maybe not relevant though since I'm not sure if there are multiaddrs where /http is valid when combined with /p2p/.

[MarcoPolo]

Could this logic get into trouble if say /ws/tls is being used for a relay and combined with /http?

/http represents the HTTP transport. It shouldn't be combined with /ws.

Maybe not relevant though since I'm not sure if there are multiaddrs where /http is valid when combined with /p2p/.

Not yet, but these could be combined. I just need to finish peer id auth over http and we could have a valid multiaddr that looks like /.../http/p2p/12Foo

[aschmahmann]

I might be missing something but given this is under the "libp2p" command I wonder if it's worth explicitly mentioning that this will enable setting up a standard HTTP proxy that uses multiaddr but otherwise doesn't use any libp2p specs at all (i.e. listening on an http multiaddr).

[MarcoPolo]

It does HTTP over streams in the spec defined way?

[aschmahmann]

Maybe this is obvious, but can you give me some context on the value of "if the endpoint is HTTPS, let's add a self-signed cert on the unix socket proxy" rather not doing it at all or having the self-signed cert being a separate flag

[MarcoPolo]

Imagine you run:

$ vole libp2p http --socket-path=/tmp/vole.sock /dns/google.com/https

What do you expect the curl request to look like?

I would expect:

curl --unix-socket /tmp/vole.sock https://www.google.com

Not

curl --unix-socket /tmp/vole.sock http://www.google.com

https:// vs http://

The only way we can do that is if our proxy itself is an https server, and thus we need a cert. Self signed is easiest. So we actually need to add a -k to the curl command.

[aschmahmann]

What do you expect the curl request to look like?

I guess this is probably my issue. Because I'm having trouble seeing the value of the HTTP -> HTTP proxy other than to demonstrate that it's possible via libp2p multiaddrs the need to drop an s vs to add a -k doesn't mean a ton to me.

On the other hand it doesn't really hurt, so as long as it's not causing trouble and you think it's useful I don't really mind

[MarcoPolo]

Because I'm having trouble seeing the value of the HTTP -> HTTP proxy other than to demonstrate that it's possible via libp2p multiaddrs

This is not the main use case, but it's nice that it works consistently. You can make the exact same curl request to a standard HTTP server as you would to a libp2p node over a stream.

[aschmahmann]

Seems good enough for now, certainly for more in-depth debugging of a specific endpoint. Wonder if supporting some basic curl like functionality would be good here so people can do one-line http-over-libp2p requests. WDYT?

[MarcoPolo]

maybe a simple GET request, and anything else would require the unix socket proxy? Does the unix socket proxy trick even work on windows?

Implementing the GET request sounds good to me, but how about we do that in a future PR once the next go-libp2p version is released? That one includes code that will make this easier. User's could make a request to multiaddr:/dns4/example.com/http/http-path/subpath and we can just pass that to go-libp2p.

[MarcoPolo]

For context, I used this when playing around with the caddy server to test its HTTP over libp2p streams ability