Text Assertions

All text assertions are deterministic — no LLM calls.

`contains(text, case_sensitive=True)`

Checks that the output contains a substring.


expect.contains("Paris")
expect.contains("paris", case_sensitive=False)

`not_contains(text)`

Checks that the output does not contain a substring.


expect.not_contains("London")

`matches_regex(pattern)`

Checks that the output matches a regular expression.


expect.matches_regex(r"\d{4}")  # contains a 4-digit number

`valid_json(schema=None)`

Checks that the output is valid JSON. Optionally validates against a JSON schema.


expect.valid_json()
expect.valid_json(schema={
    "type": "object",
    "properties": {
        "name": {"type": "string"},
        "age": {"type": "integer"}
    },
    "required": ["name", "age"]
})

`length_between(min_words, max_words)`

Checks that the output word count is within a range.


expect.length_between(10, 100)

`starts_with(prefix)`


expect.starts_with("The")

`ends_with(suffix)`


expect.ends_with(".")

`is_not_empty()`


expect.is_not_empty()

`one_of(options)`

Checks that the output exactly matches one of the given options.


expect.one_of(["yes", "no", "maybe"])

`similar_to(reference, threshold=0.7)`

Word overlap similarity (Jaccard). No LLM call.


expect.similar_to("Paris is the capital of France", threshold=0.7)

`contains_all(items)`

Must contain all substrings.


expect.contains_all(["Paris", "France", "capital"])

`contains_any(items)`

Must contain at least one substring.


expect.contains_any(["Paris", "Lyon", "Marseille"])

`structured_output(model)`

Validates output as a Pydantic model.


from pydantic import BaseModel
 
class CityInfo(BaseModel):
    name: str
    country: str
    population: int
 
expect.structured_output(CityInfo)