Skip to content

Commit 7cbeed0

Browse files
achingbrainSgtPooki2color
authored
feat: allow use as a singleton (#413)
Adds a singleton version of VerifiedFetch to make it even easier to get started: ```TypeScript import { verifiedFetch } from '@helia/verified-fetch' const response = await verifiedFetch('ipfs://Qmfoo') console.info(await response.json()) ``` --------- Co-authored-by: Russell Dempsey <1173416+SgtPooki@users.noreply.github.com> Co-authored-by: Daniel Norman <1992255+2color@users.noreply.github.com>
1 parent cb9dc90 commit 7cbeed0

4 files changed

Lines changed: 143 additions & 91 deletions

File tree

README.md

Lines changed: 56 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -13,56 +13,49 @@
1313
1414
# About
1515

16-
`@helia/verified-fetch` is a library that provides a fetch-like API for fetching trustless content from IPFS and verifying it.
16+
`@helia/verified-fetch` provides a [fetch](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API)-like API for retrieving content from the [IPFS](https://ipfs.tech/) network.
1717

18-
This library should act as a replacement for the `fetch()` API for fetching content from IPFS, and will return a [`Response`](https://developer.mozilla.org/en-US/docs/Web/API/Response) object that can be used in a similar manner to the `fetch()` API. This means browser and HTTP caching inside browser main threads, web-workers, and service workers, as well as other features of the `fetch()` API should work in a way familiar to developers.
18+
All content is retrieved in a [trustless manner](https://www.techopedia.com/definition/trustless), and the integrity of all bytes are verified by comparing hashes of the data.
1919

20-
Exports a `createVerifiedFetch` function that returns a `fetch()` like API method Helia for fetching IPFS content.
20+
This is a marked improvement over `fetch` which offers no such protections and is vulnerable to all sorts of attacks like [Content Spoofing](https://owasp.org/www-community/attacks/Content_Spoofing), [DNS Hijacking](https://en.wikipedia.org/wiki/DNS_hijacking), etc.
21+
22+
A `verifiedFetch` function is exported to get up and running quickly, and a `createVerifiedFetch` function is also available that allows customizing the underlying [Helia](https://helia.io/) node for complete control over how content is retrieved.
23+
24+
Browser-cache-friendly [Response](https://developer.mozilla.org/en-US/docs/Web/API/Response) objects are returned which should be instantly familiar to web developers.
2125

2226
You may use any supported resource argument to fetch content:
2327

24-
- CID instance
28+
- [CID](https://multiformats.github.io/js-multiformats/classes/cid.CID.html) instance
2529
- IPFS URL
2630
- IPNS URL
2731

28-
## Example
32+
## Example - Getting started
2933

3034
```typescript
31-
import { createVerifiedFetch } from '@helia/verified-fetch'
32-
33-
const fetch = await createVerifiedFetch({
34-
gateways: ['https://mygateway.example.net', 'https://trustless-gateway.link']
35-
})
35+
import { verifiedFetch } from '@helia/verified-fetch'
3636

37-
const resp = await fetch('ipfs://bafy...')
37+
const resp = await verifiedFetch('ipfs://bafy...')
3838

3939
const json = await resp.json()
4040
```
4141

4242
## Example - Using a CID instance to fetch JSON
4343

4444
```typescript
45-
import { createVerifiedFetch } from '@helia/verified-fetch'
45+
import { verifiedFetch } from '@helia/verified-fetch'
4646
import { CID } from 'multiformats/cid'
4747

48-
const fetch = await createVerifiedFetch({
49-
gateways: ['https://mygateway.example.net', 'https://trustless-gateway.link']
50-
})
51-
5248
const cid = CID.parse('bafyFoo') // some image file
53-
const response = await fetch(cid)
49+
const response = await verifiedFetch(cid)
5450
const json = await response.json()
5551
```
5652

5753
## Example - Using IPFS protocol to fetch an image
5854

5955
```typescript
60-
import { createVerifiedFetch } from '@helia/verified-fetch'
56+
import { verifiedFetch } from '@helia/verified-fetch'
6157

62-
const fetch = await createVerifiedFetch({
63-
gateways: ['https://mygateway.example.net', 'https://trustless-gateway.link']
64-
})
65-
const response = await fetch('ipfs://bafyFoo') // CID for some image file
58+
const response = await verifiedFetch('ipfs://bafyFoo') // CID for some image file
6659
const blob = await response.blob()
6760
const image = document.createElement('img')
6861
image.src = URL.createObjectURL(blob)
@@ -71,23 +64,43 @@ document.body.appendChild(image)
7164

7265
## Example - Using IPNS protocol to stream a big file
7366

67+
```typescript
68+
import { verifiedFetch } from '@helia/verified-fetch'
69+
70+
const response = await verifiedFetch('ipns://mydomain.com/path/to/very-long-file.log')
71+
const bigFileStreamReader = await response.body.getReader()
72+
```
73+
74+
## Configuration
75+
76+
### Custom HTTP gateways and routers
77+
78+
Out of the box `@helia/verified-fetch` uses a default set of [trustless gateways](https://specs.ipfs.tech/http-gateways/trustless-gateway/) for fetching blocks and [HTTP delegated routers](https://specs.ipfs.tech/routing/http-routing-v1/) for performing routing tasks - looking up peers, resolving/publishing [IPNS](https://docs.ipfs.tech/concepts/ipns/) names, etc.
79+
80+
It's possible to override these by passing `gateways` and `routers` keys to the `createVerifiedFetch` function:
81+
82+
## Example - Configuring gateways and routers
83+
7484
```typescript
7585
import { createVerifiedFetch } from '@helia/verified-fetch'
7686

7787
const fetch = await createVerifiedFetch({
78-
gateways: ['https://mygateway.example.net', 'https://trustless-gateway.link']
88+
gateways: ['https://trustless-gateway.link'],
89+
routers: ['http://delegated-ipfs.dev']
7990
})
80-
const response = await fetch('ipns://mydomain.com/path/to/very-long-file.log')
81-
const bigFileStreamReader = await response.body.getReader()
91+
92+
const resp = await fetch('ipfs://bafy...')
93+
94+
const json = await resp.json()
8295
```
8396

84-
### Configuration
97+
### Usage with customized Helia
8598

86-
#### Usage with customized Helia
99+
For full control of how `@helia/verified-fetch` fetches content from the distributed web you can pass a preconfigured Helia node to `createVerifiedFetch`.
87100

88-
You can see variations of Helia and js-libp2p configuration options at <https://helia.io/interfaces/helia.index.HeliaInit.html>.
101+
The [helia](https://www.npmjs.com/package/helia) module is configured with a libp2p node that is suited for decentralized applications, alternatively [@helia/http](https://www.npmjs.com/package/@helia/http) is available which uses HTTP gateways for all network operations.
89102

90-
The `@helia/http` module is currently in-progress, but the init options should be a subset of the `helia` module's init options. See <https://github.com/ipfs/helia/issues/289> for more information.
103+
You can see variations of Helia and js-libp2p configuration options at <https://helia.io/interfaces/helia.index.HeliaInit.html>.
91104

92105
```typescript
93106
import { trustlessGateway } from '@helia/block-brokers'
@@ -111,28 +124,28 @@ const resp = await fetch('ipfs://bafy...')
111124
const json = await resp.json()
112125
```
113126

114-
### Comparison to fetch
127+
## Comparison to fetch
115128

116-
First, this library will require instantiation in order to configure the gateways and delegated routers, or potentially a custom Helia instance. Secondly, once your verified-fetch method is created, it will act as similar to the `fetch()` API as possible.
129+
This module attempts to act as similarly to the `fetch()` API as possible.
117130

118131
[The `fetch()` API](https://developer.mozilla.org/en-US/docs/Web/API/fetch) takes two parameters:
119132

120133
1. A [resource](https://developer.mozilla.org/en-US/docs/Web/API/fetch#resource)
121134
2. An [options object](https://developer.mozilla.org/en-US/docs/Web/API/fetch#options)
122135

123-
#### Resource argument
136+
### Resource argument
124137

125-
This library intends to support the following methods of fetching web3 content from IPFS:
138+
This library supports the following methods of fetching web3 content from IPFS:
126139

127140
1. IPFS protocol: `ipfs://<cidv0>` & `ipfs://<cidv0>`
128141
2. IPNS protocol: `ipns://<peerId>` & `ipns://<publicKey>` & `ipns://<hostUri_Supporting_DnsLink_TxtRecords>`
129142
3. CID instances: An actual CID instance `CID.parse('bafy...')`
130143

131-
As well as support for pathing & params for item 1&2 above according to [IPFS - Path Gateway Specification](https://specs.ipfs.tech/http-gateways/path-gateway) & [IPFS - Trustless Gateway Specification](https://specs.ipfs.tech/http-gateways/trustless-gateway/). Further refinement of those specifications specifically for web-based scenarios can be found in the [Web Pathing Specification IPIP](https://github.com/ipfs/specs/pull/453).
144+
As well as support for pathing & params for item 1 & 2 above according to [IPFS - Path Gateway Specification](https://specs.ipfs.tech/http-gateways/path-gateway) & [IPFS - Trustless Gateway Specification](https://specs.ipfs.tech/http-gateways/trustless-gateway/). Further refinement of those specifications specifically for web-based scenarios can be found in the [Web Pathing Specification IPIP](https://github.com/ipfs/specs/pull/453).
132145

133-
If you pass a CID instance, we assume you want the content for that specific CID only, and do not support pathing or params for that CID.
146+
If you pass a CID instance, it assumes you want the content for that specific CID only, and does not support pathing or params for that CID.
134147

135-
#### Options argument
148+
### Options argument
136149

137150
This library does not plan to support the exact Fetch API options object, as some of the arguments don't make sense. Instead, it will only support options necessary to meet [IPFS specs](https://specs.ipfs.tech/) related to specifying the resultant shape of desired content.
138151

@@ -177,15 +190,15 @@ Some in-flight specs (IPIPs) that will affect the options object this library su
177190
4. [IPIP-0328: JSON and CBOR Response Formats on HTTP Gateways](https://specs.ipfs.tech/ipips/ipip-0328/)
178191
5. [IPIP-0288: TAR Response Format on HTTP Gateways](https://specs.ipfs.tech/ipips/ipip-0288/)
179192

180-
#### Response types
193+
### Response types
181194

182195
This library's purpose is to return reasonably representable content from IPFS. In other words, fetching content is intended for leaf-node content -- such as images/videos/audio & other assets, or other IPLD content (with link) -- that can be represented by <https://developer.mozilla.org/en-US/docs/Web/API/Response#instance_methods>. The content type you receive back will depend upon the CID you request as well as the `Accept` header value you provide.
183196

184197
All content we retrieve from the IPFS network is obtained via an AsyncIterable, and will be set as the [body of the HTTP Response](https://developer.mozilla.org/en-US/docs/Web/API/Response/Response#body) via a [`ReadableStream`](https://developer.mozilla.org/en-US/docs/Web/API/Streams_API/Using_readable_streams#consuming_a_fetch_as_a_stream) or other efficient method that avoids loading the entire response into memory or getting the entire response from the network before returning a response to the user.
185198

186199
If your content doesn't have a mime-type or an [IPFS spec](https://specs.ipfs.tech), this library will not support it, but you can use the [`helia`](https://github.com/ipfs/helia) library directly for those use cases. See [Unsupported response types](#unsupported-response-types) for more information.
187200

188-
##### Handling response types
201+
#### Handling response types
189202

190203
For handling responses we want to follow conventions/abstractions from Fetch API where possible:
191204

@@ -194,12 +207,12 @@ For handling responses we want to follow conventions/abstractions from Fetch API
194207
- For plain text in utf-8, you would call `.text()`
195208
- For streaming response data, use something like `response.body.getReader()` to get a [`ReadableStream`](https://developer.mozilla.org/en-US/docs/Web/API/Streams_API/Using_readable_streams#consuming_a_fetch_as_a_stream).
196209

197-
##### Unsupported response types
210+
#### Unsupported response types
198211

199212
- Returning IPLD nodes or DAGs as JS objects is not supported, as there is no currently well-defined structure for representing this data in an [HTTP Response](https://developer.mozilla.org/en-US/docs/Web/API/Response). Instead, users should request `aplication/vnd.ipld.car` or use the [`helia`](https://github.com/ipfs/helia) library directly for this use case.
200213
- Others? Open an issue or PR!
201214

202-
#### Response headers
215+
### Response headers
203216

204217
This library will set the [HTTP Response](https://developer.mozilla.org/en-US/docs/Web/API/Response) headers to the appropriate values for the content type according to the appropriate [IPFS Specifications](https://specs.ipfs.tech/).
205218

@@ -209,13 +222,13 @@ Some known header specifications:
209222
- <https://specs.ipfs.tech/http-gateways/trustless-gateway/#response-headers>
210223
- <https://specs.ipfs.tech/http-gateways/subdomain-gateway/#response-headers>
211224

212-
#### Possible Scenarios that could cause confusion
225+
### Possible Scenarios that could cause confusion
213226

214-
##### Attempting to fetch the CID for content that does not make sense
227+
#### Attempting to fetch the CID for content that does not make sense
215228

216229
If you request `bafybeiaysi4s6lnjev27ln5icwm6tueaw2vdykrtjkwiphwekaywqhcjze`, which points to the root of the en.wikipedia.org mirror, a response object does not make sense.
217230

218-
#### Errors
231+
### Errors
219232

220233
Known Errors that can be thrown:
221234

0 commit comments

Comments
 (0)