Gateway Browser UX - Loading screens?

[tmladek]

Loading an IPFS object often takes a non-trivial amount of time, and in my experience, resolving an IPNS name can even take up to half a minute.

Even a couple seconds of loading time is enough to make people unfamiliar with IPFS think that the link (URL) is broken, especially because they get no indication that the request will actually resolve - they only see a blank page in their browser.

I've repeatedly ran into this problem when I used IPFS to host a landing page of a project I was presenting at a virtual event recently. I've received dozens of e-mails complaining that "the link doesn't work", or simply that it "doesn't load" - but when I inquired further, every single time it turned out that everything was working fine on their end, the gateway just took a bit longer to load up than they're used to from regular webpages, so they abandoned it altogether. We eventually had to put up a notice about that next to the link (which most people didn't notice, so the e-mails kept coming...)

I think that if there was a generic loading page that would just display some sort of a "loading" indication (possibly with a link to learn about IPFS?) and then redirect the user to the content once it's fetched would fix this issue completely, and possibly help adoption of IPFS in general.

Now of course doing this the simplest way would break most apps that rely on the URLs actually resolving to the content represented by that hash (instead of a html/javascript document) but I think there's ways to resolve this - namely, the behavior could be dependent on the User-Agent, or it could reside on a different URL pattern (i.e. not /ipfs/{hash} but /ipfs_loading/{hash} or somesuch).

Finally, you might be right to suggest that:

• This is actually solving the symptom instead of the problem (IPFS performance).
• This will not apply to the majority of IPFS loads, and is superfluous.
• It could be worked around by using a custom/private gateway.
• The functionality has no place in the core IPFS implementation.

But I think that:

• Even if IPFS performance improves significantly, there will still be the odd and rarely-visited CIDs that will take a while to load up (at first).
• I think this could be engaged only if the gateway detects that the request takes longer than some amount of time (5 seconds?).
• Even private gateways could benefit from this (because of the above, unless the gateway has literally all requested CIDs already available)
• go-ipfs already provides a fancy (and pretty!) Web UI, so one more view wouldn't add too much weight :)

What do you think? Thanks for your consideration!

[welcome]

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review.
In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment.
Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

• "Priority" labels will show how urgent this is for the team.
• "Status" labels will show if this is ready to be worked on, blocked, or in progress.
• "Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

[Stebalien]

This has been proposed before, but suggestions to change the URL are new and interesting. We could try to add some form of ?loading=... query parameter, but may not work so well when following links.

Both cases would return a 408 Timeout response with body that includes a small script to display the progress (and reload when ready).

I'm skeptical about relying on things like the user agent (or changing the base domain). That would break downloads, for example.

A service worker could also be used to solve the issue, but we'd need some way to install it on the first request.

[ivan386]

@tmladek You can use data url or identity hash for including loading progress page in link itself.

---

Data url

data:text/html;,Loading...<meta http-equiv="refresh" content = "0; url=https://ipfs.io/ipfs/QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQw" />

---

Identity hash

Build

>ipfs add --inline --inline-limit 200000 -w --stdin-name "index.html"
ipfs: Reading from /dev/stdin; send Ctrl-z to stop.
Loading...<meta http-equiv="refresh" content = "0; url=/ipfs/QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQw" />
^Z
 113 B / ? [------------------------------------------------------------------=----------------------------------------------------------------------------------------------------------------] �
added bafyaa6iko4eaeetrjrxwczdjnzts4lrohrwwk5dbebuhi5dqfvsxc5ljoy6se4tfmzzgk43ieiqgg33oorsw45bahuqcemb3eb2xe3b5f5uxazttf5iw2ysgjvvwkmkllbyw4wlzijbfo6ccg42e4nddgvjue3skjvlec2knjzjggr3vgz4dcqlxkf3
seibphygqugdr index.html
 113 B / ? [------------------------------------------------------------------=----------------------------------------------------------------------------------------------------------------] �
added bafyabfabckgqcct5afyaa6iko4eaeetrjrxwczdjnzts4lrohrwwk5dbebuhi5dqfvsxc5ljoy6se4tfmzzgk43ieiqgg33oorsw45bahuqcemb3eb2xe3b5f5uxazttf5iw2ysgjvvwkmkllbyw4wlzijbfo6ccg42e4nddgvjue3skjvlec2knjzj
ggr3vgz4dcqlxkf3seibphygqugdrcifgs3temv4c42dunvwbq6ikaieac
 113 B / 113 B [========================================================================================================================================================================] 100.00%

Test:

https://ipfs.io/ipfs/bafyabfabckgqcct5afyaa6iko4eaeetrjrxwczdjnzts4lrohrwwk5dbebuhi5dqfvsxc5ljoy6se4tfmzzgk43ieiqgg33oorsw45bahuqcemb3eb2xe3b5f5uxazttf5iw2ysgjvvwkmkllbyw4wlzijbfo6ccg42e4nddgvjue3skjvlec2knjzjggr3vgz4dcqlxkf3seibphygqugdrcifgs3temv4c42dunvwbq6ikaieac/

[tmladek]

We could try to add some form of ?loading=... query parameter, but may not work so well when following links.

Right, hence why I thought it could reside on a different prefix (/ipfs_loading/), so at least relative links would work still. But I see it's not a small change.

That would break downloads, for example.

Oh, good point. Image loading too, most likely. You'd have to be clever with where you apply this and where you don't, and you don't really want to be "clever".

A service worker could also be used to solve the issue, but we'd need some way to install it on the first request.

I think this should be possible actually, and maybe even possible without changing IPFS - granted, the first request would still take a long time, but if that could be worked around somehow...

@ivan386 Wow, that's some dark magic! Too bad you can't fit much in the link without it blowing up in size, but I'll keep this trick in mind.

[Stebalien]

You can technically do that, but please don't. For one, it behave poorly for users who use the subdomain gateway feature:

1. User A loads http://gateway/ipfs/longcid/....
2. User A will be redirected to http://shortcid.ipfs.gateway/.... In this step, the gateway will internally:
   1. Un-inline the data in longcid, hash it, then use the resulting short CID.
3. User A will send user B a link to http://shortcid.ipfs.gateway/.
4. User B will try to load http://shortcid.ipfs.gateway/ which will likely fail because the special inlined block was never a part of the original dataset (i.e., nobody has a copy of it).

[ivan386]

@Stebalien

1. User A will be redirected to http://shortcid.ipfs.gateway/.... In this step, the gateway will internally:

   1. Un-inline the data in longcid, hash it, then use the resulting short CID.

Where i can test this case?

I only found one public gateway that failed to redirect to subdomain because cid is long:

CID incompatible with DNS label length limit of 63: k2x4le1vo7jef7qqxr56905rvrtxhgmc8z8w15x48vevyvujl24kivsvkp1wn9934495jhigerllqsdvxkjglx0ncn8xfl6rjzch8pfrxmi3begv4uay0qj7qy7ibtl4qd5s5wj0qr10adwacmzmzegql81yh8qo520mnlibt88iucpypycqte36spn3ccc92j9xv38nph9zmwypfhfjfij0zmp7vd2338elkyw9tfhxvolizyncmcj2f3pvqel0k2fych99xbgrhz1bavqkct2qwl8iuwi6bpmbulqethadj5

But it not Un-inline cid as i can see.

@tmladek
Advanced "Loading..." page is not limited by size and sources.

Example link with advanced "Loading..." page

https://gateway.ipfs.io/ipfs/uAXAAvgESKwoiEiAlhDPg8TzWOaVuHPzfk4rEUcGBGsV4FiASMkju4GXJvhIFLmh0bWwSigEKfAFVAHhMb2FkaW5nLi4uPG1ldGEgaHR0cC1lcXVpdj1yZWZyZXNoIGNvbnRlbnQ9MDt1cmw9L2lwZnMvUW1iRk1rZTFLWHFuWXlCQld4Qjc0TjRjNVNCbkpNVkFpTU5SY0d1NngxQXdRdz48aWZyYW1lIHNyYz0uaHRtbD4SCmluZGV4Lmh0bWwKAggB/

It show simple "Loading..." page and replace it with advanced "Loading..." page when it will be loaded from IPFS. At the same time it try to redirect to test link /ipfs/QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQw (this block not exist) and fail after some time.

Another example link redirects to /ipns/en.wikipedia-on-ipfs.org

https://gateway.ipfs.io/ipfs/uAXAApwESKwoiEiAlhDPg8TzWOaVuHPzfk4rEUcGBGsV4FiASMkju4GXJvhIFLmh0bWwSdApmAVUAYkxvYWRpbmcuLi48bWV0YSBodHRwLWVxdWl2PXJlZnJlc2ggY29udGVudD0wO3VybD0vaXBucy9lbi53aWtpcGVkaWEtb24taXBmcy5vcmc-PGlmcmFtZSBzcmM9Lmh0bWw-EgppbmRleC5odG1sCgIIAQ/

I make this links on this page https://ivan386.github.io/js-loading-page-generator/

But it can be done in command line:

>ipfs add -Q --raw-leaves --inline --inline-limit 200 -w --stdin-name "index.html"
ipfs: Reading from /dev/stdin; send Ctrl-z to stop.
Loading...<meta http-equiv=refresh content=0;url=/ipfs/QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQw><iframe src=.html>^Z
bafyabeybckgacct4afkqa6cmn5qwi2lom4xc4lr4nvsxiyjanb2hi4bnmvyxk2lwhvzgkztsmvzwqiddn5xhizlooq6tao3vojwd2l3jobthgl2rnvremtllmuyuwwdrnzmxsqsck54eenzujy2ggnktijxeutkwifuu2tssmndxkntygfaxoulxhy6gsztsmfwwkidtojrt2ltiorwwypqsbjuw4zdfpaxgq5dnnqmhqcqcbaaq

>ipfs add -Q --raw-leaves -r "advanced loading page"
QmQs6iRgSqix8dSgW3ooLDVXFRuoW9qLppVeY3ARnsjFx1

>ipfs object patch add-link bafyabeybckgacct4afkqa6cmn5qwi2lom4xc4lr4nvsxiyjanb2hi4bnmvyxk2lwhvzgkztsmvzwqiddn5xhizlooq6tao3vojwd2l3jobthgl2rnvremtllmuyuwwdrnzmxsqsck54eenzujy2ggnktijxeutkwifuu2tssmndxkntygfaxoulxhy6gsztsmfwwkidtojrt2ltiorwwypqsbjuw4zdfpaxgq5dnnqmhqcqcbaaq .html QmQs6iRgSqix8dSgW3ooLDVXFRuoW9qLppVeY3ARnsjFx1|ipfs cid format -b base64url
uAXAAwwESLgoiEiAlhDPg8TzWOaVuHPzfk4rEUcGBGsV4FiASMkju4GXJvhIFLmh0bWwYnQoSjAEKfAFVAHhMb2FkaW5nLi4uPG1ldGEgaHR0cC1lcXVpdj1yZWZyZXNoIGNvbnRlbnQ9MDt1cmw9L2lwZnMvUW1iRk1rZTFLWHFuWXlCQld4Qjc0TjRjNVNCbkpNVkFpTU5SY0d1NngxQXdRdz48aWZyYW1lIHNyYz0uaHRtbD4SCmluZGV4Lmh0bWwYeAoCCAE

[Stebalien]

Where i can test this case?

Hm. You're right. We were going to redirect to a "short" cid but I guess we decided not to implement that (probably for the reasons I mentioned). So I guess right now, we'd just fail.

[ivan386]

@tmladek

Loading an IPFS object often takes a non-trivial amount of time, and in my experience, resolving an IPNS name can even take up to half a minute.

It problem of the root block. When we change some content on website it change root hash. Even if other content of website is same we need to fetch this root block from original source.

I think a little about this problem. I already solve this problem on one of my websites by identity hash.

Dnslink of my website contain identity hash of block that contains catalog with links to "index.html" and "content" catalog.

Content catalog contain all website data. Hash of it can often change. On other hand index.html does not change. And it will be loaded almost immediately because many peers already have it block.

Identity hash for that website can be build like this:

>ipfs add --raw-leaves --inline --inline-limit 100 -w -Q index.html
bafyaaoysgufciakvciqekxlv3epi5shx5da6nhvexpxymzxtpj5cicbcqadurgp2e3oqahqsbjuw4zdfpaxgq5dnnqmiwbykaieac

>ipfs add --raw-leaves -r -Q content
QmQs6iRgSqix8dSgW3ooLDVXFRuoW9qLppVeY3ARnsjFx1

>ipfs object patch add-link bafyaaoysgufciakvciqekxlv3epi5shx5da6nhvexpxymzxtpj5cicbcqadurgp2e3oqahqsbjuw4zdfpaxgq5dnnqmiwbykaieac content QmQs6iRgSqix8dSgW3ooLDVXFRuoW9qLppVeY3ARnsjFx1|ipfs cid format -b base64url
uAXAAbRIwCiISICWEM-DxPNY5pW4c_N-TisRRwYEaxXgWIBIySO7gZcm-Egdjb250ZW50GJ0KEjUKJAFVEiBFXXXZHo7I9-jB5p6ku--GZvN6eiQIIoAHSJn6Jt0AHhIKaW5kZXguaHRtbBiLBwoCCAE

That final identity hash i write in dnslink record of website.

@Stebalien Can block contain ipns links?

[lidel]

Closing this, as UX is mostly addressed:

• Kubo 0.37 shipped timeout of 30s: feat(config): Gateway.RetrievalTimeout|MaxConcurrentRequests #10905
• Kubo 0.38 will ship improved UX showing why timeout occured: feat(gateway): improved error page with retrieval state details #10950
• (TBD) Kubo 0.39 may lower the timeout to something closer to 5-15s

If you feel UX could be improved, please fill a new issue (this one is 5 year old).