Safety

We can tell you the mechanism, not just the promise.

Four things happen between a child typing a sentence and anything coming back. Here they are, in order.

What actually happens

1. The screen runs before the model

Every sentence a student types is checked for a self-harm signal first. If one fires, the sentence never reaches the AI at all — the conversation stops, the child is shown real help, and the school’s safeguarding lead is alerted. This is server-side, so no phrasing talks its way past it.

2. The tutor refuses the answer

The companion knows which assignment is open and graded. On one of those it will not produce a worked solution. It offers a ladder of hints instead, and the student is told what the next rung costs before they take it. If it cannot verify what a question belongs to, it fails closed and treats it as graded.

3. Nothing parent-visible is unreviewed

A note home, a progress report, a parent-meeting brief — all drafts, all waiting on a teacher pressing approve on a different screen. The AI cannot write into a child’s record. It proposes; a human commits.

4. A child sees their own work and nobody else’s

Tenancy is checked on every query, and a student tool reads the learner’s identity from their session, never from what the page sent. A child asking about another child is not a permission we have — it is a shape the API cannot express.

The one exception, said out loud

Machine marking does not wait for a teacher, and that is a decision rather than an oversight.

When a student submits a quiz it is marked immediately — objectively where it can be, against a stored rubric where it cannot. That total reaches the marks register without a teacher in the loop.

We think the alternative is worse for the child. Holding every score until a teacher is at a desk means no student ever finds out how they did on the evening they did it.

So the human path is correction rather than permission: every proposed mark carries a confidence score and a record of who marked it, so an unreviewed one is identifiable; a teacher can re-mark any answer; and the attempt total and the register recompute when they do. Self-practice a student sets themselves never enters the register at all.

Safeguarding alerts cannot be switched off

Every other notification in the product is a preference. This one is not, and the settings screen says so on the row rather than rendering a switch.

A parent or teacher who has muted the one message that says a child may be in trouble is a worse outcome than any amount of notification fatigue — and offering the toggle at all implies it is a reasonable thing to do.

We red-team it, and we write down what failed

Four adversarial runs so far, the most recent on the student tool shelf.

Each run tries to make the product do the thing it promises not to: get the answer out of the tutor, reach another child’s work, get a self-harm sentence past the screen, publish something to a parent without approval. The record lists every failure found, what was changed, and which risks were accepted rather than fixed.

The most recent run found five issues and blocked release until three of them were fixed — among them a child’s private practice set being published to their whole class, titled with their weakest skill. That is the kind of thing a run exists to catch, and the kind of thing a vendor who does not run one never finds.

One risk is still open on the register. We will show you the record, including that item, on a call — ask for it at your demo. A safety page with no open items is a safety page nobody audited.

Where the data goes

Written plainly, because you will find out in procurement anyway.

• Files you upload are stored in Mumbai.
• Some AI processing currently runs outside India. We would rather say that here than have you discover it later.
• Prompts carry first names and the work itself. Full names, contact details and identifiers do not go to the model.
• Nothing is used to train any model, ours or anyone else’s.
• A parent can withdraw AI consent for their child at any time, per purpose, from their own settings. The tutor keeps working; it simply stops remembering.
• Read the privacy page for the full picture — it is in draft, under legal review against the DPDP Act 2023.