upstream-bleve/docs/custom_query.md

# Custom Queries: `custom_filter` and `custom_score`

Bleve exposes two query nodes for embedding-defined per-hit logic:

- `custom_filter`: keep or drop each candidate hit.
- `custom_score`: mutate each candidate hit score.

Bleve itself only executes callbacks that were already provided/attached by the
embedding application. It does not interpret any
embedder-specific payload such as callback source, "params", or requested fields.

## Query Objects

`CustomFilterQuery` contains:

- a child query
- a `searcher.CustomFilterFunc` callback attached by the embedder

`CustomScoreQuery` contains:

- a child query
- a `searcher.CustomScoreFunc` callback attached by the embedder

## Constructors

Embedding applications can construct the nodes directly with callbacks:

```go
filterQuery := query.NewCustomFilterQueryWithFilter(childQuery,
    func(d *search.DocumentMatch) bool {
        return true
    },
    nil, // fields; nil when callback does not need field values
    nil, // payload; nil when no source/params round-trip is needed
)

scoreQuery := query.NewCustomScoreQueryWithScorer(childQuery,
    func(d *search.DocumentMatch) float64 {
        return d.Score
    },
    nil, // fields
    nil, // payload
)
```

Constructor arguments after the callback are:

- `fields`: field names to preload into `d.Fields` via doc values before the callback runs
- `payload`: non-query custom attributes that must survive JSON round-trips (for example distributed fan-out)

Pass `nil` for either when not needed:

```go
payload := map[string]interface{}{
    "params": map[string]interface{}{"weight": 0.05},
    "source": "function score(doc, params) { ... }",
}

scoreQuery := query.NewCustomScoreQueryWithScorer(childQuery,
    func(d *search.DocumentMatch) float64 {
        f := d.Fields
        return d.Score + (f["abv"].(float64) * 0.05)
    },
    []string{"abv", "ibu"},
    payload,
)
```

Payload rules:

- pass only non-query custom-node attributes like `source` and `params`
- do not include `query` in payload; `query` always comes from the child query argument
- do not include `fields` in payload; `fields` come from the dedicated constructor argument
- pass `nil` as the fourth argument when payload round-trip is not needed

## Field Access and Doc Values Requirement

When the `fields` constructor argument is non-empty, bleve loads the listed
field values into `DocumentMatch.Fields` before invoking the callback. Field
values are read from **doc values** (column-oriented, memory-mapped), not
stored fields.

**Doc values must be enabled for every field listed in `"fields"`.** If a field
does not have doc values enabled in the index mapping, its value will be silently
absent from `DocumentMatch.Fields` at callback time — no error is raised, the
field simply won't appear. This can cause filters to drop all hits or scores to
compute incorrectly if the callback assumes the field is present.

Doc values are enabled by default for all field types (`DocValues: true`) and
for dynamic mappings (`DocValuesDynamic: true`), so no extra configuration is
needed unless these defaults have been explicitly disabled. If they have been
disabled, re-enable them:

```go
fm := mapping.NewTextFieldMapping()
fm.DocValues = true                    // default is true; only needed if previously disabled
imap.DefaultMapping.AddFieldMappingsAt("genre", fm)
```

For dynamic mappings, ensure `docvalues_dynamic` has not been set to `false` on
the document mapping.

**Datetime fields require an explicit datetime mapping.** Dynamic mapping indexes
date-like strings as text, which means the standard analyzer tokenizes the value
(e.g. `"2010-07-22 20:00:20"` becomes `["2010", "07", "22", ...]`). The doc
value then contains tokens instead of the original date, making comparisons
fail. To use datetime fields in custom_filter or custom_score callbacks, add an
explicit field mapping with `"type": "datetime"` in the index definition.

### Field type decoding

At the bleve layer, doc values are decoded based on the field's mapping type:

| Mapping type | Go type in `d.Fields` | Notes |
| --- | --- | --- |
| `text` / `keyword` | `string` | Raw text bytes |
| `number` | `float64` | IEEE 754 decoded |
| `boolean` | `bool` | `true` / `false` |
| `datetime` | `float64` | Epoch nanoseconds stored as a floating-point value |

## Runtime Behavior

- Bleve builds the child query searcher first.
- If `fields` is non-empty, a `DocValueReader` is created for those fields.
- `custom_filter` wraps the child with `CustomFilterSearcher`.
- `custom_score` wraps the child with `CustomScoreSearcher`.
- On each hit, doc values are loaded into `DocumentMatch.Fields`, then the callback is invoked.
- The callback receives only `*search.DocumentMatch`; it does not receive `SearchContext`.

## Error Cases

- missing child query:
  - `"custom filter query must have a query"`
  - `"custom score query must have a query"`
- missing bound callback:
  - `"custom filter query must have a filter callback"`
  - `"custom score query must have a score callback"`