Comparisons / Test harnesses
RowShield vs pg_prove: running suites versus watching deployments
The short version
- pg_prove is the Perl-based runner that executes pgTAP suites and reports TAP results in pipelines. RowShield is not a runner at all — it is scheduled, centralised verification of live Postgres posture requiring no suite.
- Choose pg_prove when — you already maintain pgTAP suites and need them wired into CI reliably.
- Choose RowShield when — the suite keeps breaking or was abandoned, and you want posture findings, drift alerts and remediation SQL without authoring another assert.
RowShield rules relevant here
Head to head: pg_prove vs RowShield
| Capability | pg_prove | RowShield | Edge |
|---|---|---|---|
| Role in the pipeline | Executes whatever tests exist and reports the tally; output quality depends entirely on suite upkeep, which the runner cannot see. | Generates findings directly from the live catalog on every scan — there is no suite to write, update or resurrect. | RowShield |
| Target environment | Whichever database the job points at — usually an ephemeral rebuild from migrations, discarded again when the run ends. | The production and staging projects themselves, connected once and scanned on schedule whether or not anyone pushed code today. | RowShield |
| Signal on completion | An exit code plus TAP text inside job logs — accurate for whoever still opens that log. | Transition alerts to chat, email or webhook, plus a CLI whose exit codes (0/1/2) give pipelines parity with runners. | Parity |
| Behavioural probing | None. A runner executes assertions; it never issues requests as the anon caller to learn what PostgREST actually serves. | Anon-key GET probes show what PostgREST serves today, including the filtered-versus-empty distinction fixture tests flatten away. | RowShield |
| Ecosystem fit | Native to Perl and TAP toolchains many platform teams already operate; slots into those pipelines without adaptation. | A dashboard, CLI and API; beyond Node for the CLI there are no runtime dependencies and nothing to install on hosts. | pg_prove |
| Cost profile | Open source and free; the continuing price is engineer time keeping suites aligned with evolving schemas. | Free tier available for daily scans; paid plans add tighter cadence and longer history. | pg_prove |
| Drift detection between runs | Only if something re-runs the suite between incidents — exactly the discipline that lapses first when delivery pressure rises. | Core behaviour: every scan diffs the previous snapshot, so drift detection happens by construction rather than by vigilance. | RowShield |
Column claims about pg_prove are sourced below. Where the edge is theirs, the page says so — and the sections that follow explain why Supabase teams still pick RowShield.
What pg_prove does
As part of the pgTAP ecosystem, pg_prove connects to a database, runs .sql test files through the TAP protocol, and fails the job when asserts fail. In disciplined hands it is exactly right: deterministic, scriptable, honest.
Mechanically it is a TAP source handler: hand it test scripts and connection details, and it dispatches each file to the server, parses the returned TAP stream into pass and fail counts, and sets its exit code from the result. Wiring it up means curating file order, credentials and which suites count as merge gates.
Discipline is the operative word. Runners do not write tests, update them, or notice when the suite stops matching production ambitions.
Where the scopes differ
A runner answers "did the asserts pass"; RowShield answers "what is true right now". The first is bounded by suite quality; the second reads the catalog directly, nine rule classes deep, and tracks every finding across runs.
When pipelines go red because a month-old assert no longer matches a legitimate refactor, teams mute the job. Muted jobs are indistinguishable from absent ones during the next real incident.
Supabase sharpens the mismatch because posture changes through surfaces git never sees: the dashboard policy editor, the SQL console, provider-side restores, agent-generated patches applied directly. A runner observes none of these events; scheduled catalog diffing exists to observe precisely them.
Why Supabase teams choose RowShield over pg_prove
Zero authoring, zero rot, production scope. Findings carry severity and copy-paste remediation SQL; alerts fire on transitions so quiet projects stay quiet; and the same engine ships as a CLI with clean exit semantics (0 clean, 1 findings at threshold, 2 broken run) if you still want a pipeline gate.
Migration from a stalled runner setup is unceremonious: connect the project, let the first scan build the baseline, and route alerts where the old job's logs used to go. Teams describe the shift as trading archaeology — reconstructing why a suite went red — for reading a finding that names the table, the rule and the fix.
Where pg_prove is the right choice
If your organisation standardises on TAP reporting and maintains its suites properly, pg_prove is free and proven. Keep it for general database invariants — constraints, functions, migrations — where its generality wins and our fixed rule set has nothing to add.
It also suits teams whose databases are genuinely disposable per run, since a runner over an ephemeral rebuild is the natural shape there. The pivot arrives when the database stops being disposable: once real users depend on one long-lived project, verification has to follow that project, which is RowShield's whole design centre.
Using both
Common pattern: pgTAP suites for engine-level invariants in CI, RowShield for continuous posture and drift in production. The rowshield CLI exit codes slot beside pg_prove jobs without conflict, so a pipeline can keep both gates — asserts for logic, scans for deployed reality — and reviewers read a single combined signal instead of two philosophies.
Frequently asked
- Is RowShield affiliated with the pgTAP project?
- No. Veristria builds RowShield independently; pgTAP and pg_prove belong to their maintainers and are referenced descriptively here. Both are respected tools with long histories, and nothing on this page implies endorsement or partnership. Where we describe how they behave, we rely on their public documentation rather than private knowledge of either project.
- Is RowShield a good pg_prove replacement?
- For RLS and exposure verification, yes — no suite required, because findings come from the catalog itself. For executing bespoke database tests, pg_prove remains the tool; the two coexist happily, and many teams keep their engine-level asserts in CI while handing posture and drift to scheduled scans against the deployed project.
- Does RowShield integrate with CI?
- Yes — the CLI exits 0 when clean, 1 at or above your fail-on threshold, and 2 when the scan itself could not run, so pipelines can distinguish unsafe schemas from broken scanners. Scheduled monitoring continues independently of CI, meaning coverage persists even for weeks when nobody touches the repository at all.
Check your project in about ten seconds
Paste a URL. No signup, no writes, nothing stored.
Run the free audit