How a decision is made

Pointsman decides which way a case goes. A case is anything a system wants decided: a report of a closed road, a new issue, a pull request. Each kind of case has a profile: the questions to ask, the facts to look up and the rules that turn the answers into an action. A profile is versioned, so every decision can say which rules made it.

How a decision is made

1. A case comes in. A system sends it to Pointsman over the HTTP API, or an AI agent sends it over MCP (the Model Context Protocol, which AI agents use to call tools). A connector, a part that links another system to Pointsman, can also bring it in, for example from a FIWARE smart-city data platform.
2. Questions and facts, at the same time.
   • An AI model answers the profile’s questions: yes or no, one choice from a list, or a score. Each answer comes with a probability: how sure the model is, from 0 to 1.
   • Pointsman looks up the profile’s facts in data, for example whether the place is in a flood zone or how many files a pull request changes. The AI does not see or guess facts.
3. Rules decide. People write the rules, for example “danger 70 % or more: urgent”. Pointsman checks them in order; the first rule whose condition is true (the first that fits) gives the action. When no rule fits, the profile’s default action applies. An action is a name the profile chooses, such as auto, urgent or review.
4. An action, or a person. The action goes back to the system, which acts on it. When the action is review (or another action the profile lists for people to check), a person checks the case first, gives the final action and can correct the AI’s answers.
5. Everything is recorded in the decision log, with the corrections, which show how often the AI was right.

Why it is split this way

• The AI answers, people decide. The model only answers narrow questions. What follows from the answers is written in rules that people can read, test and change. So the model can be replaced without changing what the profile does, and a person can always see why a decision was made. Some decisions need no AI answer at all: a profile can approve a release pull request on facts alone, when it changes only version fields and the changelog and other facts check who made it (see github_release in the profile format).
• Probabilities, not only answers. A rule can ask how sure the model is, for example team.p >= 0.85 (p is the probability of the answer to the question team). When the model is not sure, the rules send the case to a person instead of guessing.
• Facts are looked up, not guessed. Things that can be measured, like a distance or the files of a pull request, come from data. When a fact cannot be looked up, it is missing, never made up; see Missing facts.
• One record for each decision. Because the log keeps the profile version and the model, you can later ask what a changed rule would have decided, and how often it would have agreed with people.

Missing facts

A lookup can fail: the case has no location, the data service is slow, or a pull request got a new commit while Pointsman was reading it. Then the fact is missing. Pointsman never fills it in with a guess: a guessed value looks like a real one, and a rule would treat it as true.

Every fact has two more fields besides its own: missing is true when the fact has no value, and reason says why, for example timeout or no_location (the reasons are listed under Fact types).

A rule that compares a missing fact, or a field without a value, is neither true nor false: it is unknown, and a rule fits only when its condition is true. false and unknown is false, true or unknown is true, and not unknown stays unknown. So a rule never fits because of data that was not there. But a missing fact can stop an earlier rule that blocks something, so that a later rule that allows it fits instead. That is why a profile tests missing first, which is always known:

- when: "facts.flood.missing == true"
  action: review

reason tells a failed lookup from one that does not apply: a detour can only be computed for a road section, so facts.walk.missing == true and facts.walk.reason != 'no_location' sends only a real failure to a person. In a rule that allows something, name every condition as a known value (facts.risky.any == false) instead of counting on an earlier rule. The details are in the profile format.

AI answers only make a decision stricter

Some cases are written by the very person the decision is about: a pull request’s title and description come from its author, a report from whoever sent it. That person can write text meant to steer the AI, for example “this change is safe”. This is called prompt injection, and no model is fully safe from it.

So in a profile whose input comes from outsiders, the profile’s author follows a rule: an AI answer may only make the decision stricter. It can send a case to a person or ask for a fix, but it is never the reason a case passes. What must be true for a pass is checked with facts, values that Pointsman works out itself and the author of the case cannot set directly. Pointsman does not enforce this rule: a profile that passes or approves on AI answers alone can be steered by whoever writes the case. The example profile workflow-check, shortened:

policy:
  rules:
    - when: "facts.wf.missing == true"
      action: retry
    # ...
    - when: "touches_secrets.yes >= 0.3"
      action: review
    - when: >-
        facts.wf.unparsed == 0 and facts.wf.privileged == 0
        and facts.wf.injection == 0 and facts.wf.unpinned == 0
        and touches_secrets.yes < 0.3
      action: pass
  default: review

The AI question touches_secrets reads the pull request’s text. It appears in a rule that sends the case to a person, and in the pass rule only as one more condition. A text that fools the AI can at most remove that extra condition; the facts still have to allow the pass. When the facts cannot be read, the first rule gives retry (try again later), never pass.

Before you publish such a profile, read its rules as sentences (node scripts/explain-profile.mjs <file>) and check that:

1. every rule that lets a case pass names the facts it relies on;
2. AI answers appear only in rules with a stricter action, or as an extra condition in a pass rule;
3. a missing fact leads to a person or to retry, never to a pass.

People: reviews and corrections

A profile lists the actions a person checks before they count (by default review). Such a decision waits in the review queue; a person gives the final action and can correct the AI’s answers. The client can pass a callback_url to receive the final answer, signed, when the person is done. Corrections are kept as feedback, so you can measure how often a profile was right. Details: reviews and callbacks.

The decision log

Every decision is stored before it is returned: if the log cannot be written, the client gets an error and no decision, so nobody acts on a decision that is not recorded. The log keeps the answers, the facts, the rule, the model, the versions of the engine and its modules, and the profile version, which says which questions were asked. The request itself is kept only as a hash, unless the profile asks to keep it. A test token’s decisions are not logged. Some facts read the log too, for example how many of an author’s pull requests wait for a review; they see only the asking client’s own decisions. Details: decision log.

Decision chains

A decision can build on earlier ones: a report is checked first, more checks run, and a final decision reads all of them (requires in the profile). The earlier decisions are named by id and read from the log, so a client cannot make up their results. Details: decision chains.

How Pointsman is built

Pointsman is one engine with parts around it. Each deployment chooses the parts it needs, so a city’s road-closure service and a team’s pull-request checks run the same engine with different modules.

• Engine. Takes a case, asks the AI model the profile’s questions, gets the facts, applies the rules and writes the decision log.
• AI models answer the questions. A profile names a model and fallbacks; they can be swapped without changing the rules (models).
• Fact modules look up facts: flood zones and shelters (@pointsman/facts-gsi), the way around a closed road (@pointsman/facts-valhalla), the state of a pull request (@pointsman/github) or its AI reviews (@pointsman/coderabbit). They are packages in this repository, built into the engine; a deployment lists the ones it uses (fact modules).
• Connectors connect another system: they bring its cases in and carry the results back, for example to a FIWARE platform (@pointsman/fiware) or to a pull request (@pointsman/github). They use the same public API as any other client (connectors). The FIWARE connector, its actions and settings are described in the FIWARE bridge, and the demo it runs in the FIWARE demo storyboard.
• Adapters connect the engine to the platform it runs on: where the log and the profiles are stored, which AI service answers, how requests come in. Today that is Cloudflare Workers, Cloudflare’s service for running code on its servers; the engine core uses no Cloudflare-specific code (adapters).

One package can have more than one role: @pointsman/github looks up facts and is a connector. Keeping the parts apart keeps the engine small: a new data source or a new system is a new module, not a change to the engine, and another platform needs other adapters while the profiles stay the same.

Where to go next

• Your first profile shows all of this on your own computer.
• Profile format has every field.
• Security says what Pointsman protects, and against whom.