# Search backends, and OpenSearch

2026-08-08. Asked for directly, mid-build: *"Um OpenSearch as well."*

Today all search is MySQL `FULLTEXT`, asked through the content-type registry.
That is genuinely fine for most sites and it is staying. What this adds is a
**seam**, and one implementation of it against OpenSearch.

---

## 🚨 It has to be optional, and that constrains everything

Convoro's whole proposition is that you upload a zip and it runs. No Composer,
no build step, no runtime Node, and — the web installer exists precisely for
this — no shell. An OpenSearch cluster is a Java service somebody has to
operate.

So:

- **The database backend stays the default and stays complete.** Not a
  degraded mode, not a fallback nobody tests. It is what a fresh install uses
  and what most sites will keep.
- **Nothing about OpenSearch may leak into the applications.** Blog, Gallery,
  Forums and the rest already say how to find their own content; they must not
  learn that a second backend exists.
- **If OpenSearch is switched on and then falls over, search still works.**
  A community whose search box breaks because a container was restarted is a
  worse outcome than slightly weaker relevance.

That last one is the difference between a driver seam and a dependency.

## The shape

A `SearchIndex` contract with two implementations, chosen by a setting:

- **`DatabaseIndex`** — today's behaviour exactly. Asks the content-type
  registry's `find` callbacks. Always available, indexes nothing, needs nothing.
- **`OpenSearchIndex`** — queries a cluster over HTTP with hard timeouts, and
  **reports itself unavailable rather than throwing** the moment anything is
  wrong. The service above it then uses the database.

🚨 The fallback is not a `catch` around the whole search page. It is a
capability check plus a per-call failure result, because "the cluster answered
with nonsense" and "the cluster is down" must both end at the same place: the
database, quietly, with the visitor getting results.

## 🚨 Permissions are the hard part, and the obvious answer is wrong

Search is a permission boundary. This codebase has already leaked through it
once — a page restricted to administrators returned 404 when opened and handed
a signed-out visitor its title through the search box.

Every database `find()` filters by the viewer's groups **in SQL**. An external
index cannot do that, because the filter has to be in the query sent to the
cluster.

**The obvious answer: store the allowed group ids on each document.** It is
wrong, and the reason is worth writing down.

Restrictions change *after* documents are indexed. Restrict a forum that has
ten thousand topics, and ten thousand documents are now stale — and stale in
the dangerous direction: they still say "everybody may see this". Until a
reindex catches up, the search box is handing out titles from a forum somebody
has just made private. A reindex is minutes at best, and a job queue nobody
noticed had stopped at worst.

**What is done instead: documents carry their container, and the *viewer's*
readable containers are resolved at query time.** Every document has a `gate` —
`forum:12`, `blog:3`, `album:7`, or nothing at all for content that is public
by nature. At query time the search service asks each content type which gates
this viewer may read — which is a database question about *current*
restrictions, the same one the application already answers — and filters the
query to that set.

The consequence is the point: **restricting a forum takes effect on the next
search, with no reindexing at all.** The index never holds a permission, so it
can never hold a stale one.

The cost is a list of ids in each query. That list is bounded by the number of
containers on the site — tens, occasionally hundreds — not by the amount of
content, which is the number that actually grows.

🚨 A document with no gate is readable by everybody, and that default is
deliberate — but it means **a content type that forgets to set a gate publishes
its restricted content**. The registry therefore treats a missing `gates`
callback as "this type is not indexable", rather than as "this type has no
restrictions". Forgetting produces absence, not disclosure.

## Indexing belongs to the task runner

🚨 The standing rule is no outbound call inside a visitor's request. Search
itself is the third deliberate exception, on the same terms as checkout and the
assistant: the call *is* what the visitor asked for, with hard timeouts and a
visible fallback.

**Indexing is the opposite case and gets no exception.** Writing a document to
a cluster while somebody waits for their post to save is exactly the shape of
call the rule exists to prevent. So content changes mark work to be done and
the task runner does it, and a full rebuild is a command.

## What this note does not decide

- **Faceting, aggregations and "did you mean".** The reason people want
  OpenSearch, and worth having — but a backend that returns the same results as
  the database one, only better ranked, is the honest first step. Faceting
  changes the results *page*, which is its own piece of work.
- **Per-type index tuning** — analysers, synonyms, language stemming. Defaults
  first, measured against real questions.
- **Elasticsearch.** The API surface used here is small and mostly shared, but
  claiming support for something never tested is how "compatible" becomes a
  support burden. One backend, named honestly.
