feat: filter pubs using jsonata

[tefkah]

Issue(s) Resolved

High-level Explanation of PR

While brainstorming with @3mcd on how the sitebuilder could function differently, it became clear that the easiest way to make the sitebuilding more flexible would be to specifiy some sort of query for most fields.

The original idea was to use JSONATA for this, but we would need to have some way of translating this to SQL to actually filter Pubs from the db.

Behold!

Basically a reimplementation of #985 but in a JSONata-y style.

You can do stuff like

$contains($lowercase($.pub.title = "croc"))

and get all Pubs whose title match croc.

You can also match on values, relations, and even incoming relations.

Eg say you have a Book called "Future Book of Knowledge", with a PubField starter:chapters, which relates to chapters.

Say you want all chapters of that book after chapter 3, you could do

$.pub.in.chapters[$.relatedPub.title = "Future Book of Knowledge"] and $.pub.values.chapter-number > 3 

Pub Resolver

I also replaced how we are doing Pub resolving automations with this. @3mcd was already doing most of this.

It mostly uses the above querying language now, with one modification: you need to be explicit about which JSONata is meant to be interpolated.

Previously this was sort of hard to understand in the resolver, as the left hand side would be interpolated from the incoming data, while the right hand side acted more like the querying and would not be interpolated.

I tried to make this a little more explicit by requiring you to use {{ }} around the stuff you want interpolated. This is kinda weird, like JSONata inside JSONata, but I think this is the most clear, double braces always meaning you enter a new interpolation context.

Prior art

• Mostly Erics work on the Pub resolver!
• GROQ from Sanity. Also a very simple syntax that can do quite a lot.

shoutout claude 4.5 opus

Test Plan

See tests!

Screenshots (if applicable)

Notes

[Copilot]

Pull request overview

This PR introduces JSONata-based query syntax for filtering pubs, providing a more flexible and expressive alternative to the existing filter system. The implementation includes both SQL generation for database queries and in-memory filtering capabilities.

Changes:

• Adds JSONata query parser and compiler that converts JSONata expressions to SQL and in-memory filters
• Implements support for comparisons, logical operators, string functions, and relation queries
• Updates automation resolver to use JSONata syntax with explicit {{ }} interpolation blocks

Reviewed changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
core/package.json	Adds jsonata dependency
core/lib/server/pub.ts	Adds customFilter option to getPubsWithRelatedValues for JSONata-based filtering
core/lib/server/jsonata-query/types.ts	Defines AST and internal representation types for JSONata query parsing
core/lib/server/jsonata-query/sql-builder.ts	Implements SQL generation from parsed JSONata conditions
core/lib/server/jsonata-query/parser.ts	Parses JSONata expressions into internal condition format
core/lib/server/jsonata-query/memory-filter.ts	Implements in-memory filtering using parsed conditions
core/lib/server/jsonata-query/jsonata-query.db.test.ts	Comprehensive test suite for parser, SQL generation, and database queries
core/lib/server/jsonata-query/index.ts	Exports public API for JSONata query functionality
core/lib/server/jsonata-query/errors.ts	Custom error classes for query parsing and validation
core/lib/server/jsonata-query/compiler.ts	Compiles JSONata expressions to queryable format
core/lib/server/jsonata-query/SYNTAX.md	Documentation for supported JSONata query syntax
core/app/c/[communitySlug]/stages/manage/components/panel/automationsTab/StagePanelAutomationForm.tsx	Updates automation resolver UI with improved design and help text
core/actions/_lib/resolveAutomationInput.ts	Refactors resolver to use JSONata queries with explicit interpolation syntax

Files not reviewed (1)

• pnpm-lock.yaml: Language not supported

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The builtin field list in line 198 is missing 'title' and 'stageId', which are included in the BUILTIN_FIELDS constant (lines 103-110). This inconsistency means queries using $.relatedPub.title or $.relatedPub.stageId will fail even though these are valid builtin fields.

Suggested change

	if (["id", "createdAt", "updatedAt", "pubTypeId"].includes(thirdStep.value)) {
	return {
	kind: "relatedPubBuiltin",
	field: thirdStep.value as "id" | "createdAt" | "updatedAt" | "pubTypeId",
	if (
	["id", "createdAt", "updatedAt", "pubTypeId", "title", "stageId"].includes(
	thirdStep.value
	)
	) {
	return {
	kind: "relatedPubBuiltin",
	field: thirdStep.value as
	| "id"
	| "createdAt"
	| "updatedAt"
	| "pubTypeId"
	| "title"
	| "stageId",

[Copilot]

The PubsId import was removed but it's still being used in the function signature on line 189 as communityId parameter and in the old code references. This appears to be an oversight from the refactoring, though the code still compiles because the type is inferred elsewhere.

Suggested change

import type { CommunitiesId } from "db/public"

[Copilot]

The prefixTerms variable (derived from user input via cleanQuery and terms) is directly interpolated into the SQL template. Although cleanQuery removes some special characters, prefix matching with :* could be exploited. Consider using parameterized queries or additional sanitization for the tsquery input.

[Copilot]

The MAX_RELATION_DEPTH constant should have a comment explaining why this limit exists (likely performance or complexity concerns) to help future maintainers understand the rationale.

Suggested change

	// Limit how deeply relation paths can be nested in a single query. This keeps
	// query evaluation and validation predictable and avoids the performance and
	// complexity costs of arbitrarily deep relation traversal.

[Copilot]

The parseJsonataQuery function is synchronous but it's being wrapped in Promise.resolve. This is unnecessary and adds complexity. Either make parseJsonataQuery async or call it directly without tryCatch wrapping a promise.

Suggested change

	const [parseError, _parsedQuery] = await tryCatch(
	Promise.resolve(parseJsonataQuery(interpolatedExpression))
	)
	let parseError: Error | null = null
	try {
	parseJsonataQuery(interpolatedExpression)
	} catch (error) {
	parseError = error as Error
	}

[Copilot]

The date string detection on line 97 uses v.includes('-') which is too broad and will incorrectly parse strings like 'hello-world' as dates if they happen to parse successfully. Consider using a more specific date format check or regex pattern.

[Copilot]

The searchArg is directly interpolated into the LIKE pattern string without escaping. User input containing special LIKE characters (%, _) could lead to unintended pattern matching. Consider escaping these characters before interpolation.

[Copilot]

The new imports are not in alphabetical order. The Textarea import should come after Select and before TokenProvider, and the Tooltip imports should follow the established ordering pattern in this file.

Suggested change

	import { FormSubmitButton } from "ui/submit-button"
	import { Textarea } from "ui/textarea"
	import { Textarea } from "ui/textarea"
	import { FormSubmitButton } from "ui/submit-button"