# Classifying with a Model

Ask the built-in typesafe_jev model typed questions, and wrap it in a tool that classifies a card purchase and records the decision on the transaction.

A model sends a state and typed questions to a provider's model and returns typed answers: a choice among options, a score on a scale, or the probability of yes. Twisp ships every model, so there is nothing to create. In this tutorial you read the `typesafe_jev` model, call it directly, and then wrap it in a tool that reads a card purchase from the ledger, asks the model about it, and records the decision on the transaction.

> **Note:**
>
> You need a running [local environment](https://www.twisp.com/docs/infrastructure/local-environment.md) that holds a TypeSafe API key. Open GraphiQL in the local console and run each step as its own request. If you have run this tutorial before, choose fresh UUIDs, account and tran code codes, and action names.

The model's state and questions leave Twisp on every call: they are sent to TypeSafe, which answers. Send only the data the questions need.

## 1. Read the model

`typesafe_jev` is an action of kind `model` that every tenant has. List the models:

```graphql
query ReadModels {
  actions {
    actions(where: { native: true, kind: "model" }, first: 50) {
      nodes {
        actionId
        version
        status
        definition {
          name
          enabled
          config
          tool {
            name
            inputSchema
          }
        }
      }
    }
  }
}
```

```json
{
  "nodes": [
    {
      "actionId": "83d17bdc-ece0-59e6-9af3-6dbb76ea351d",
      "version": 1,
      "status": "ACTIVE",
      "definition": {
        "name": "typesafe_jev",
        "enabled": true,
        "config": { "api": "decision/1", "model": "jev-1.13.0", "provider": "typesafe" },
        "tool": { "name": "typesafe_jev", "inputSchema": { "...": "..." } }
      }
    }
  ]
}
```

`config.model` is the TypeSafe release this version calls, and `inputSchema` requires `state` and `questions`, where each question is a `choice`, `score`, or `noul`. The `actionId` is the same in every tenant. Note the `version`: a call names the exact version it runs, and each version of a model calls one release.

## 2. Ask it questions

`invokeAction` calls the model synchronously. The input is a `state`, which can be a string, an object, an array, or null, and a map of `questions` keyed by an ID you choose. This call asks three questions about one card purchase:

- `category` is a `choice`: each key of `criteria` is an option, described by its value. A description can be null, as for `other`.
- `review` is a `score`: `criteria` lists the levels in order, from level 0.
- `personal` is a `noul`: a yes-or-no question.

```graphql
mutation AskJev {
  actions {
    invokeAction(
      id: "83d17bdc-ece0-59e6-9af3-6dbb76ea351d"
      input: {
        executionId: "b40afa17-1aeb-4a97-a4db-26080741c877"
        actionVersion: 1
        input: {
          state: {
            merchant: "SQ *BLUE BOTTLE COFFEE OAKLAND CA"
            amount: "86.40 USD"
            card: "Engineering corporate card"
          }
          questions: {
            category: {
              type: "choice"
              instructions: "Which expense category fits this card purchase?"
              criteria: {
                meals: "Food, coffee, or restaurants."
                travel: "Airfare, hotels, rideshare, or fuel."
                software: "Subscriptions, cloud services, or licenses."
                other: null
              }
            }
            review: {
              type: "score"
              instructions: "How closely should finance review this purchase?"
              criteria: ["No review needed.", "Spot check.", "Full review with receipts."]
            }
            personal: {
              type: "noul"
              instructions: "Is this likely a personal purchase rather than a business expense?"
            }
          }
        }
      }
    ) {
      executionId
      status
      invocation {
        output
      }
    }
  }
}
```

The execution is `SUCCEEDED`, and its output is the model's answers, keyed by question ID:

```json
{
  "executionId": "b40afa17-1aeb-4a97-a4db-26080741c877",
  "status": "SUCCEEDED",
  "invocation": {
    "output": {
      "answers": {
        "category": {
          "type": "choice",
          "choice": "meals",
          "confidence": 1,
          "probabilities": { "meals": 1, "other": 0, "software": 0, "travel": 0 }
        },
        "personal": { "type": "noul", "noul": 0.39 },
        "review": {
          "type": "score",
          "score": 0.95,
          "confidence": 0.47,
          "legend": {
            "0": "No review needed.",
            "1": "Spot check.",
            "2": "Full review with receipts."
          },
          "probabilities": { "0": 0.2, "1": 0.65, "2": 0.15 }
        }
      },
      "model": "jev-1.13.0",
      "usage": { "input_tokens": 462, "output_tokens": 77 }
    }
  }
}
```

Each answer has the shape of its question's type:

| Type     | Fields                                           | Reading it                                                                                                     |
| -------- | ------------------------------------------------ | -------------------------------------------------------------------------------------------------------------- |
| `choice` | `choice`, `probabilities`, `confidence`          | `choice` is the chosen option; `probabilities` has one entry per option.                                       |
| `score`  | `score`, `legend`, `probabilities`, `confidence` | `score` is a position on the scale, from 0 to the top level; `legend` maps each level number to its criterion. |
| `noul`   | `noul`                                           | The probability that the answer is yes.                                                                        |

`model` records the release that answered, which is always the release the version names, and `usage` the tokens the call used. The answers are a model's judgment, so the numbers you see can differ from these, and can change from one call to the next.

## 3. See an error

The input schema holds the provider's rules, so Twisp refuses a malformed question before it leaves Twisp. A `score` level needs a description, so a scale of null levels is refused:

```graphql
mutation AskJevBadly {
  actions {
    invokeAction(
      id: "83d17bdc-ece0-59e6-9af3-6dbb76ea351d"
      input: {
        executionId: "d36e8004-4157-45be-8860-5a9df7d84e37"
        actionVersion: 1
        input: {
          state: "A purchase."
          questions: { review: { type: "score", criteria: [null, null] } }
        }
      }
    ) {
      status
      invocation {
        error {
          code
          message
          hint
          details
        }
      }
    }
  }
}
```

The call fails with `action_input_invalid`, and the message names the path of each problem. The levels at `criteria/0` and `criteria/1` are null:

```json
{
  "status": "FAILED",
  "invocation": {
    "error": {
      "code": "action_input_invalid",
      "message": "tool input: at '/questions/review': 'oneOf' failed, none matched; ... at '/questions/review/criteria/0': got null, want string or array or object; at '/questions/review/criteria/1': got null, want string or array or object; ...",
      "hint": "...",
      "details": null
    }
  }
}
```

A request that passes the schema can still be refused by the provider. That call fails with `model_request_invalid`, and `details.provider` carries the provider's message.

A failed call is still recorded, with its input and error. The call is made once: retrying is the caller's decision. The [reference](https://www.twisp.com/docs/reference/actions/models.md#errors) lists every model error.

## 4. Post a card purchase

The tool you build next reads a purchase from the ledger. Create a card account, a settlement account, and a tran code whose transaction description is the merchant, then post one purchase:

```graphql
mutation SetUpCardPurchase {
  card: createAccount(
    input: {
      accountId: "f717eaef-2856-4986-92a7-4df8a5c434f6"
      code: "CLS-CARD"
      name: "Engineering corporate card"
      normalBalanceType: DEBIT
    }
  ) {
    accountId
  }
  settlement: createAccount(
    input: {
      accountId: "fd71bd5e-3daa-433b-bbea-5c4fdb1ced72"
      code: "CLS-SETTLEMENT"
      name: "Card settlement"
      normalBalanceType: CREDIT
    }
  ) {
    accountId
  }
  purchase: createTranCode(
    input: {
      tranCodeId: "e93301fc-41b7-40f8-8575-27e60c2bb860"
      code: "CLS_CARD_PURCHASE"
      description: "A purchase on a corporate card."
      params: [
        { name: "card", type: UUID }
        { name: "amount", type: DECIMAL }
        { name: "merchant", type: STRING }
      ]
      transaction: { description: "params.merchant" }
      entries: [
        {
          accountId: "params.card"
          units: "params.amount"
          currency: "'USD'"
          entryType: "'CARD_PURCHASE_DR'"
          direction: "DEBIT"
          layer: "SETTLED"
        }
        {
          accountId: "uuid('fd71bd5e-3daa-433b-bbea-5c4fdb1ced72')"
          units: "params.amount"
          currency: "'USD'"
          entryType: "'CARD_PURCHASE_CR'"
          direction: "CREDIT"
          layer: "SETTLED"
        }
      ]
    }
  ) {
    tranCodeId
  }
  postTransaction(
    input: {
      transactionId: "71488c02-3739-4f12-b190-3863194a4274"
      tranCode: "CLS_CARD_PURCHASE"
      params: {
        card: "f717eaef-2856-4986-92a7-4df8a5c434f6"
        amount: "86.40"
        merchant: "SQ *BLUE BOTTLE COFFEE OAKLAND CA"
      }
    }
  ) {
    transactionId
    description
    metadata
  }
}
```

The transaction's `description` is the merchant, and its `metadata` is empty.

## 5. Create the review tool

The tool takes a transaction ID. Its program reads the transaction and its entries with `graphql`, builds the state from the merchant, amount, and card, asks the model, and writes the decision into the transaction's metadata.

The questions are fixed in the tool's reference to the model. Fixed input maps a top-level input property to a CEL expression, here a map literal, and replaces whatever the caller supplies, so the program passes only `state` and every call asks exactly these questions. The reference pins version 1, so a new release of the model changes nothing until you move the pin. The program calls the model by its name, `tools.typesafe_jev`.

```graphql
mutation CreateReviewTool {
  actions {
    createAction(
      id: "da1b191c-2113-40f6-add7-2afc76fd0953"
      input: {
        name: "models-review-purchase"
        description: "Categorize a card purchase and flag it for review"
        kind: "tool"
        tool: {
          name: "review_purchase"
          description: "Categorize a posted card purchase and record whether finance should review it."
          inputSchema: {
            type: "object"
            properties: { transaction_id: { type: "string", format: "uuid" } }
            required: ["transaction_id"]
            additionalProperties: false
          }
          outputSchema: { type: "object" }
        }
        tools: [
          {
            name: "typesafe_jev"
            version: 1
            input: {
              questions: """
              {
                "category": {
                  "type": "choice",
                  "instructions": "Which expense category fits this card purchase?",
                  "criteria": {
                    "meals": "Food, coffee, or restaurants.",
                    "travel": "Airfare, hotels, rideshare, or fuel.",
                    "software": "Subscriptions, cloud services, or licenses.",
                    "other": null
                  }
                },
                "review": {
                  "type": "score",
                  "instructions": "How closely should finance review this purchase?",
                  "criteria": ["No review needed.", "Spot check.", "Full review with receipts."]
                },
                "personal": {
                  "type": "noul",
                  "instructions": "Is this likely a personal purchase rather than a business expense?"
                }
              }
              """
            }
          }
        ]
        policies: [
          {
            effect: ALLOW
            actions: [SELECT]
            resources: ["financial.Entry.*", "financial.Account.*"]
          }
          {
            effect: ALLOW
            actions: [SELECT, INSERT, UPDATE]
            resources: ["financial.Transaction.*"]
          }
        ]
        config: {
          runtime: "starlark"
          src: """
          READ = '''
          query Purchase($id: UUID!) {
            transaction(id: $id) {
              description
              entries(first: 2) {
                nodes { direction amount { units currency } account { name } }
              }
            }
          }
          '''

          TAG = '''
          mutation Tag($id: UUID!, $metadata: JSON!) {
            updateTransaction(id: $id, input: { metadata: $metadata }) { transactionId }
          }
          '''

          def run(event):
              id = event["data"]["transaction_id"]
              tx = graphql(READ, {"id": id}, operation_name="Purchase")["transaction"]
              if tx == None:
                  fail("No transaction " + id, hint="Pass the transactionId of a posted purchase.")
              debit = [e for e in tx["entries"]["nodes"] if e["direction"] == "DEBIT"][0]
              state = {
                  "merchant": tx["description"],
                  "amount": debit["amount"]["units"] + " " + debit["amount"]["currency"],
                  "card": debit["account"]["name"],
              }
              result = tools.typesafe_jev({"state": state}, check=False)
              if result.ok:
                  answers = result.output["answers"]
                  decision = {
                      "category": answers["category"]["choice"],
                      "needs_review": answers["personal"]["noul"] >= 0.5 or answers["review"]["score"] >= 1.5,
                  }
              else:
                  decision = {"category": "unclassified", "needs_review": True, "error": result.error.code}
              graphql(TAG, {"id": id, "metadata": decision}, operation_name="Tag")
              return decision
          """
        }
      }
    ) {
      actionId
      version
      status
      pending
    }
  }
}
```

How the program acts on the answers:

- A purchase needs review when it is more likely personal than not, or when the review score is closer to a full review than to a spot check.
- `check=False` turns a model error into a result instead of a failure. If the model cannot answer, for example with `model_unavailable`, the purchase is still tagged, as `unclassified` and needing review, with the error code.
- Updating a transaction writes a new version of it, so the policies allow `INSERT` and `UPDATE` on transactions, besides `SELECT` on what the program reads.

Poll until the tool is live:

```graphql
query ReadReviewTool {
  actions {
    action(id: "da1b191c-2113-40f6-add7-2afc76fd0953") {
      version
      status
      pending
      definition {
        tools {
          name
          actionId
          actionVersion
        }
      }
    }
  }
}
```

Wait for `status: ACTIVE` with `pending: null`, and use its `version` in the calls below. The reference resolved `typesafe_jev` to the model's `actionId`, and its `actionVersion` is 1, the version it pins. A reference without a `version` would call the model's latest version instead.

## 6. Review the purchase

Call the tool with the transaction from step 4:

```graphql
mutation ReviewPurchase {
  actions {
    invokeAction(
      id: "da1b191c-2113-40f6-add7-2afc76fd0953"
      input: {
        executionId: "2c4d544e-b97e-4b36-b355-618fa4fef183"
        actionVersion: 2
        input: { transaction_id: "71488c02-3739-4f12-b190-3863194a4274" }
      }
    ) {
      executionId
      status
      invocation {
        output
      }
    }
  }
}
```

```json
{
  "executionId": "2c4d544e-b97e-4b36-b355-618fa4fef183",
  "status": "SUCCEEDED",
  "invocation": { "output": { "category": "meals", "needs_review": false } }
}
```

The decision is on the transaction:

```graphql
query ReviewedPurchase {
  transaction(id: "71488c02-3739-4f12-b190-3863194a4274") {
    description
    metadata
  }
}
```

```json
{
  "transaction": {
    "description": "SQ *BLUE BOTTLE COFFEE OAKLAND CA",
    "metadata": { "category": "meals", "needs_review": false }
  }
}
```

The model call and the metadata write are one unit with the request: they commit together, and if the request rolled back, neither would remain.

## 7. Read the recorded calls

Every call the tool made is a child execution: the two `graphql` calls and the model call, in the order the program made them. The model's child records its input, the state the program built merged with the fixed questions, and its output. Its transcript records the request exactly as it was sent to TypeSafe and the response exactly as it came back.

```graphql
query ReviewCalls {
  actions {
    execution(id: "2c4d544e-b97e-4b36-b355-618fa4fef183") {
      status
      children(first: 10) {
        nodes {
          callSequence
          action {
            definition {
              name
            }
          }
          status
          event {
            data
          }
          invocation {
            output
            transcript(first: 10) {
              nodes {
                sequence
                kind
                data
              }
            }
          }
        }
      }
    }
  }
}
```

```json
{
  "status": "SUCCEEDED",
  "children": {
    "nodes": [
      {
        "callSequence": 1,
        "action": { "definition": { "name": "graphql" } },
        "status": "SUCCEEDED",
        "event": { "data": { "operation_name": "Purchase", "query": "...", "variables": { "id": "71488c02-..." } } },
        "invocation": { "output": { "data": { "transaction": { "description": "SQ *BLUE BOTTLE COFFEE OAKLAND CA", "entries": { ... } } } } }
      },
      {
        "callSequence": 2,
        "action": { "definition": { "name": "typesafe_jev" } },
        "status": "SUCCEEDED",
        "event": {
          "data": {
            "questions": { "category": { ... }, "personal": { ... }, "review": { ... } },
            "state": {
              "amount": "86.40 USD",
              "card": "Engineering corporate card",
              "merchant": "SQ *BLUE BOTTLE COFFEE OAKLAND CA"
            }
          }
        },
        "invocation": {
          "output": {
            "answers": {
              "category": { "type": "choice", "choice": "meals", ... },
              "personal": { "type": "noul", "noul": 0.38 },
              "review": { "type": "score", "score": 0.92, ... }
            },
            "model": "jev-1.13.0",
            ...
          },
          "transcript": {
            "nodes": [
              {
                "sequence": 1,
                "kind": "model.request",
                "data": { "provider": "typesafe", "method": "POST", "path": "/v1/systemone", "body": "{\"model\":\"jev-1.13.0\",\"state\":{...},\"questions\":{...}}" }
              },
              {
                "sequence": 2,
                "kind": "model.response",
                "data": { "status": 200, "request_id": "...", "latency_ms": 840, "body": "{\"model\":\"jev-1.13.0\",\"answers\":{...},\"usage\":{...}}" }
              }
            ]
          }
        }
      },
      {
        "callSequence": 3,
        "action": { "definition": { "name": "graphql" } },
        "status": "SUCCEEDED",
        "event": {
          "data": {
            "operation_name": "Tag",
            "query": "...",
            "variables": {
              "id": "71488c02-...",
              "metadata": { "category": "meals", "needs_review": false }
            }
          }
        },
        "invocation": { "output": { "data": { "updateTransaction": { "transactionId": "71488c02-..." } } } }
      }
    ]
  }
}
```

The `graphql` calls' transcripts are empty, so their nodes show `"transcript": { "nodes": [] }`, omitted here. The record answers the audit questions for each decision: what data left Twisp, which release answered, what it said, and what the tool wrote because of it. The API key is never recorded.

## 8. Review a purchase that needs a look

Post a second purchase and review it in the same request. The call runs inside the request transaction, so the tool reads the transaction the request just posted:

```graphql
mutation PostAndReview {
  postTransaction(
    input: {
      transactionId: "861d2e86-5a71-467e-9f46-09cb15df2277"
      tranCode: "CLS_CARD_PURCHASE"
      params: {
        card: "f717eaef-2856-4986-92a7-4df8a5c434f6"
        amount: "59.99"
        merchant: "STEAMGAMES.COM 4259522985 WA"
      }
    }
  ) {
    transactionId
  }
  actions {
    invokeAction(
      id: "da1b191c-2113-40f6-add7-2afc76fd0953"
      input: {
        executionId: "8f5c9555-6ab2-4a47-a3e0-23d7d43e2167"
        actionVersion: 2
        input: { transaction_id: "861d2e86-5a71-467e-9f46-09cb15df2277" }
      }
    ) {
      status
      invocation {
        output
      }
    }
  }
}
```

```json
{
  "postTransaction": { "transactionId": "861d2e86-5a71-467e-9f46-09cb15df2277" },
  "actions": {
    "invokeAction": {
      "status": "SUCCEEDED",
      "invocation": { "output": { "category": "software", "needs_review": true } }
    }
  }
}
```

The model judged a game store purchase on an engineering card more likely personal than not, so the tool flagged it. Read this call's children, as in step 7, to see the answer behind the flag: in this run, `personal` was `0.74`.

## Troubleshooting

| Symptom                                                | Check                                                                                                                                                                                                                                                              |
| ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Every model call fails with `model_credential_missing` | The deployment has no TypeSafe API key. You never supply a key; the deployment holds it.                                                                                                                                                                           |
| Every model call fails with `model_not_served`         | The deployment does not offer `typesafe_jev`. Ask your Twisp operator to enable it.                                                                                                                                                                                |
| A call fails with `action_input_invalid`               | The input does not match the model's schema: `state` and `questions` are both required, a `score` needs at least two levels and none null, a `choice` at least one option, and a `noul` instructions or a `true` or `false` criterion. The message gives the path. |
| A call fails with `model_request_invalid`              | The provider refused the request. Read `details.provider` for its message.                                                                                                                                                                                         |
| A call fails with `model_unavailable`                  | The provider was rate limited, overloaded, or did not answer within 10 seconds. Call again later.                                                                                                                                                                  |
| A call fails with `action_not_callable`                | The version you call is retired. The hint names the latest version: move your pin to it.                                                                                                                                                                           |
| The review tool fails with `action_access_denied`      | The tool's policies must allow `SELECT` on what it reads and `INSERT` and `UPDATE` on transactions. The hint names the denied operation and resource.                                                                                                              |
| `invokeAction` returns `ALREADY_EXISTS`                | The `executionId` was used before with another action, version, or input. Choose a new one.                                                                                                                                                                        |
| Your answers differ from the ones shown                | Answers are a model's judgment and can vary between calls. Pin a version, as this tutorial's tool does, so a new release is a change you make.                                                                                                                     |

Read [Models](https://www.twisp.com/docs/actions/models.md) for when to use a model and how it fits with tools and agents. The [models reference](https://www.twisp.com/docs/reference/actions/models.md) covers every field, error, and transcript event, [Tool references](https://www.twisp.com/docs/reference/actions/tools.md#tool-references) covers fixed input, [Starlark](https://www.twisp.com/docs/reference/actions/starlark.md#handling-failed-calls) covers `check=False`, and [ActionExecution](https://www.twisp.com/docs/reference/actions/executions.md#action-execution) covers the recorded calls.