Autocomplete search

[evanp]

"As an ActivityPub API user, I want to have a prefix search feature available, so I can mention someone partially and let my client autocomplete the rest of their address."

[evanp]

From #20

[mediaformat]

This would be useful for Hashtags as well!

[evanp]

So, I think it makes sense to define this by its purpose (autocomplete actor reference) rather than its algorithm (prefix search).

Full actor search usually searches a lot of different fields, like the full name, bio, location, and so on, with fine-tuned options.

Prefix search is probably what we're looking for here; it's usually how you implement typeahead search. Searching for "benp" would find "@benpate@server.example" but not "@gerald@other.example" whose bio has "Reporter for the Globe And Post, see my latest article at https://globenpost.example/gerald".

But it's also quite common to see interfaces where you type in "@Prodrom<TAB>" and it fills in with "@evan@cosocial.ca" (or what have you).

So, let's try something like this: a defined endpoint for actor autocomplete:

{
    "@context": "https://www.w3.org/ns/activitystreams",
    "type": "Person",
    "id": "https://social.example/user/123",
    "inbox": "https://social.example/user/123/inbox",
    "outbox": "https://social.example/user/123/outbox",
    "followers": "https://social.example/user/123/followers",
    "following": "https://social.example/user/123/following",
    "liked": "https://social.example/user/123/liked",
    "endpoints": {
        "sharedInbox": "https://social.example/shared/inbox",
        "actorAutocomplete": "https://social.example/search/actor/typeahead"
    }
}

The endpoint takes a single parameter, q, which is pretty commonly used for "the search parameter". You fetch it with a GET request.

GET /search/actor/typeahead?q=evan HTTP/1.1
Host: social.example
Accept: application/activity+json, application/ld+json, application/json
Authorization: Bearer CJz4LwUWgn1Xfgs4TB307

It returns a collection of Actor objects.

{
    "@context":  "https://www.w3.org/ns/activitystreams",
    "type": "Collection",
    "id": "https://social.example/search/actor/typeahead?q=evan",
    "totalItems": 3,
    "items": [
         {
            "id": "https://other.example/user/C6CEFD42-2109-4719-9E71-231804076FB7",
            "type": "Person",
            "name": "Evan Prodromou",
            "preferredUsername": "evanp"
         },
         {
            "id": "https://third.example/actor/DJQ7R4Xw7mX8GWnueYPpL",
            "type": "Application",
            "name": "Evander Holyfield Quotes Bot",
            "preferredUsername": "evanderbot"
         },
         {
            "id": "https://fanclub.example/group/thewaspfans",
            "type": "Group",
            "name": "Evangeline Lilly Fan Club",
            "preferredUsername": "thewaspfans"
         }
    ]
}

Note that the order and matching algorithm are server-defined. The actors found can come from the actor's followers or following collections, from the local server user directory, from a global user directory, a Ouija board, whatever.

In the last example actor, the search parameter is in the name, not the preferredUsername. The match can be on different properties of the actor, can be exact, or prefix, or fuzzy, or whatever. It's up to the server to choose the best actors to show.

The collection can also be paged. If so, it's up to the server to make sure that the paging is stable.

If there are no actors found, the response is still returned (200 OK, not 404 Not Found!), but the items array is empty.

We can also define a couple of problem details types. It's common for this kind of search to require a minimum search parameter size, usually around 3, so one problem could be "search param too small". In the spec, we could say that 3 chars is the recommended minimum.

Here's an example where the minimum is 4 characters:

HTTP/1.1 400 Bad Request
Content-Type: application/problem+json

{
    "type": "https://swicg.github.io/activitypub-api/problems#autocomplete-search-param-too-small",
    "title": "Search param too small",
    "detail": "The q parameter must be 4 characters or more",
    "q": "ev",
    "minimum": 4
}

I think that should probably work for most autocomplete use cases?

[trwnh]

using an entire HTTP request for a single autocomplete is very wasteful and prone to high latency. you generally don't want this to be happening over a network for every mention, even with debounce.

what other apps do is preload an "address book" of some sort and locally search through that. in firefox, this is your bookmarks and history and so on. you aren't searching through a global address space, you're searching through a local subset of it.

for this to work well, we need to define a format for such an index so that clients can use it locally, then sync it between clients on-demand (with http GET and cache control headers).

in effect, this is describing a webfinger cache for acct: lookups per the mastodon or swicg/apwf profile. the format should store mappings using a line/based format, ideally, to make it easier to process. i feel like json is unnecessary overhead for this, and a plain text format would be better, but json(l?) is still an option i guess.

an almost hosts-file-like mapping (very common in unix using readline):

benpate@server.example https://benpate.server.example/

minimal mapping into RDF with text/turtle:

<acct:benpate@server.example> as:alsoKnownAs <https://benpate.server.example/>.

although if it were an AS2 document (there are reasons to keep this for "activity processors") then it could be a Collection:

{
"@context": "https://www.w3.org/ns/activitystreams",
"id": "https://autocomplete-index-collection.example/",
"items": [
{"id": "acct:benpate@server.example", "alsoKnownAs": "https://benpate.server.example/"}
]
}

this being a Collection, it could be optionally paged if it gets too big.

on the consuming side, what i would probably do if the collection was the canonical format is merge all collection pages/items into an rdf graph, discard the kind-of-useless statements about the autocomplete index collection itself, then reserialize the graph as Turtle (or TriG with the collection as context) and optionally remove the predicates and URIRef angle brackets to get the minimal hosts-like mapping (which could be loaded into something like Elasticsearch, but could also be searched directly with something like grep). even more simply, if it was jsonl, i could grep for the specific line (without loading every item in the page or every item in the collection) and parse that line as a JSON object and get the "alsoKnownAs" value.

[evanp]

Some responses on this:

• Do we need a separate discovery mechanism besides endpoints.

endpoints is an object that can be referenced with a link:

{
    "@context": "https://www.w3.org/ns/activitystreams",
    "type": "Person",
    "id": "https://social.example/user/123",
    "inbox": "https://social.example/user/123/inbox",
    "outbox": "https://social.example/user/123/outbox",
    "endpoints": "https://social.example/shared/endpoints"
}

[evanp]

Also refer to : https://helge.codeberg.page/fep/fep/844e/