DeepFakeShield — Synthetic Media Verification for Audio & Video

1. The problem

A recording arrives. It may be a voice note, a phone call, a radio capture, a CCTV clip, a conference excerpt or a piece of evidence in a disciplinary case. It sounds plausible and looks plausible, and somebody has to decide whether it can be trusted before it changes a decision.

The tools that existed before generative media are not much help. A file's metadata is trivially edited. A human listening for "robotic" delivery is being asked to detect a signal that modern synthesis is specifically engineered to remove. And the online detectors that promise an answer in seconds require uploading the very recording that must not leave the organisation — a compliance problem dressed as a convenience.

Sending a confidential recording to an external API to find out whether it is fake means the recording is now in someone else's building. For a police force, a broadcaster, an insurer, a hospital, a defence contractor or a bank, that is not an option, which leaves the decision to intuition.

DeepFakeShield exists to remove the choice between "send it away" and "guess".

---

2. What DeepFakeShield is

DeepFakeShield is an analysis product you install inside your own environment. You submit an audio file or a video; it returns a decision, the reasons behind that decision, an assessment of how good the submitted media was, and an evidence record naming the exact engine and version that produced the result. Nothing leaves the deployment. The media is deleted after analysis by default, and what remains is the record — not the recording.

It is deliberately not a black box with a percentage on it. A percentage invites a decision no one can defend. A decision state, a reason code, a quality assessment and a named engine invite a decision an organisation can stand behind, and can later show its reasoning for.

---

3. Who it is for

The common thread is not the industry. It is a recording that cannot be sent anywhere, and a decision that has to be defended later.

---

4. How it works

Seven steps, all of them inside your perimeter.

1. Ingest. A file arrives by drag-and-drop in the console, by POST to the REST API, or as a chunked/streamed short clip. Eleven container formats are accepted across audio and video. The container is sniffed from its actual bytes — a file whose extension lies is refused rather than trusted — the size limit is enforced while the bytes are still arriving, and the file is written under a randomised name. Executable content is never processed.
2. Fingerprint. The media's SHA-256 is computed on the way in. That digest is what identifies the media afterwards: the recording can be deleted safe in the knowledge that a re-submission of the same bytes can be recognised.
3. Normalise. Audio is decoded to mono PCM at a fixed sample rate through FFmpeg. Video is sampled to frames at a bounded rate, with timestamps preserved so that findings can be pointed at a moment, not just a file. If the video carries an audio track, that track is demuxed and analysed as a second, independent channel.
4. Assess quality. Before anything is concluded, the pipeline measures the media: signal-to-noise proxy, clipping, silence, speech presence, effective bandwidth for audio; blur variance, contrast, blockiness and temporal change for video. A heavily compressed call recording and a studio master are not the same evidence, and the result says which one it had.
5. Detect. Each channel is scored by the configured detector. Two backends ship: a deterministic in-process engine that runs anywhere, including CPU-only hosts with no GPU and no downloads, and an optional neural sidecar (AASIST for audio, Xception with RetinaFace for video) for GPU hosts. Which engine ran is recorded per analysis, and a model whose checksum does not match refuses to load rather than running quietly.
6. Decide. Channel scores and quality are fused under a versioned threshold profile into one of four states, and the rule that produced the outcome is recorded with it. Thresholds are configuration, never constants in code, so the operating point can be changed and every historical result still says which point it was judged under.
7. Record and dispose. An evidence record is written — media digest, metadata, engine, engine version, weights digest, configuration version, both channel scores, quality, decision, reason codes, timings, software version — and sealed with a digest the API re-verifies before serving it. The media is deleted as part of the same operation unless retention was explicitly requested.

---

5. What you get back

Every analysis returns the same structure, whatever the media and whichever engine ran.

A decision state, out of four.

Reason codes that name conditions, not vibes. Thirty-two of them, machine-readable, each with a plain-language explanation: SYNTHETIC_AUDIO_SCORE_HIGH, AUDIO_VIDEO_DISAGREEMENT, AUDIO_LOW_QUALITY, NO_FACE_DETECTED, INSUFFICIENT_SPEECH, MODEL_ERROR, UNSUPPORTED_MEDIA, and more. They are the same codes in the interface, in the REST response and in the audit trail.

A quality assessment. GOOD, ACCEPTABLE, LOW_QUALITY or INSUFFICIENT_MEDIA, with the measurements behind it. This is the field that stops a 4 kHz call recording from being read as a finding about the speaker.

An evidence record. The document an auditor, a regulator or a court asks for: what media was examined, what it hashed to, which engine and version examined it, under which configuration, what each channel scored, how long it took, which software version produced it, and a digest over the whole thing.

---

6. Privacy and data residency

Privacy is not a setting on this product; it is the shape of it.

• Nothing is uploaded anywhere. The detector and the database sit on an internal container network with no route to the internet. There is no telemetry back to Sensifai and no phone-home.
• The media is deleted, not stored. By default the file is removed as part of the operation that completes the analysis. The evidence record says the media was deleted, and says when.
• Retention is opt-in, bounded and enforced. An operator can ask for retention with a window; the expiry is written into the record and enforced by a sweep. A deployment where the sweep is doing the work is a deployment with a bug, not a deployment with a policy.
• Your media is never training data. Submitted media is not used to train anything, and no path exists for it to leave the deployment to be trained on.
• Deployment is yours. Docker Compose on your hardware, or Coolify on your own host — including a GPU host for the neural path. Bring your own database, your own identity provider, your own TLS terminator.

---

7. What it deliberately does not claim

This section is part of the product, not a disclaimer bolted onto the back of it. Every result screen carries a version of it.

It does not claim an accuracy rate. No error rate, no detection percentage, no "97 % accurate". None has been measured, and a security product that publishes an unmeasured number is telling you about its marketing rather than its behaviour. When an operating point has been measured against a labelled corpus, it will be published with the corpus named — until then the field says uncalibrated and the interface says so in plain words.

It does not claim a calibrated probability. The score is a detector score compared against a configured threshold. It is not the probability that the media is fake, and the product refuses to present it as one.

It does not explain artefacts it cannot see. Reason codes name conditions the engines actually test. No engine here identifies an artefact type — no "this face shows boundary blending" — so no field claims one.

It cannot prove authenticity. A LIKELY_GENUINE result means the evidence examined was consistent with authentic capture. It is not a certificate, and it does not survive a new manipulation technique that the configured detectors were not designed to notice.

It is not a monitoring platform. Four operator screens and a REST API — not a SOC dashboard, not a case-management system, not a real-time feed of a network. Those are different products with different requirements, and pretending otherwise would hide the ones this product does meet.

---

8. The four screens your analysts actually use

Dashboard. Analyses today, audio and video counts, suspicious and uncertain counts, average and median processing time, the live queue, the engine health, and the most recent results. Five figures and a list, because that is what an operator wants at 09:00.

New analysis. Drag and drop, or the API. Choose Auto (analyse whichever channels the file contains), Audio, Video, or Audio + video; choose the threshold profile; decide whether to retain the media. Upload progression is shown per file, with the reason a file was refused stated in the same place it was dropped.

Analysis result. The overall decision and the sentence explaining it; per-channel results with each channel's score, thresholds, engine, model version and weights digest; input quality with the measurements; every reason code with its explanation; per-segment scores for video; the operating point in force; the processing time; the media digest; and export as JSON or as a plain-text report.

Analysis history. Every analysis, searchable by reference, file name or media digest, filterable by status, decision and media kind. The columns an operator scans: ID, date, type, result, score, quality, model, processing time.

Plus an Account screen (plan, usage and API keys) and an Administration panel (users and roles, engines and models, retention, audit log, marketplace connection) for whoever runs the deployment.

---

9. The REST API, and integrating it

Everything the console does, the API does — the console is a client of the same endpoints, not a parallel implementation. That is a design commitment rather than a feature list item: a figure on the dashboard and the same figure in an API response are produced by the same code, so they cannot disagree.

Documented with OpenAPI 3.1, rendered at /api/v1/docs. The core surface:

Authentication is an API key with a fixed scope set, or a session. A key carries exactly the permissions its scopes grant and never more than its creator had; a revoked key stops working on the next request. Errors are RFC 7807 problem documents carrying a correlation id that matches the log line, so a failure in someone's integration can be found in your logs by the id they quote.

---

10. Deployment and operations

Shape. One compose file, five containers: PostgreSQL, the API (which also serves the console bundle), the worker, the scheduler, and — only if you ask for it with --profile neural — the neural detector sidecar. Three networks: one shared with your proxy, one for the application processes that need egress, and one internal-only network carrying the database and the detector, with no published host ports anywhere in the file, because the stack is reached through your proxy rather than by exposing the API on the host.

Requirements. A Linux host with Docker. 4 GB of RAM and 2 CPU cores are enough for the built-in engines on interactive workloads. A GPU host (an RTX-class card on the reference build) is what the neural path wants, and the sidecar is a separate compose overlay so a CPU host simply does not run it.

Configuration. Environment only, never a file in the image. Non-root containers, read-only filesystem where possible, secrets supplied by your orchestrator, and a deployment verifier that checks a running stack for published ports, default secrets, a reachable database, working FFmpeg, a writable media root, available engines and one end-to-end analysis before you put it in front of anyone.

Operations. Structured JSON logs with a correlation id per request and an analysis id per job; Prometheus-format metrics behind the audit permission; a self-test an administrator can run from the Engines screen that probes the database, FFmpeg, the media root, the engines, the model registry and the vocabulary; a retention sweep and a queue that recovers work from a worker that died mid-job.

Migrations. Versioned SQL, applied as a deploy step, with the schema version reported by the self-test and by /api/v1/version.

---

11. Security posture

• Authentication and authorisation. Your identity provider (Logto, or any OIDC provider) or scoped API keys. Roles — analyst, reviewer, administrator — are enforced by the API on every request; the interface hides what a role cannot reach, and the API refuses it regardless, because hiding a button is not a permission check.
• Isolation. Non-root containers, an internal-only data network, no media egress, and no executable processing anywhere in the ingestion path.
• Input handling. Extension, declared MIME type and actual byte signature must agree; size is enforced during the upload; files land under randomised names in an ephemeral location.
• Model integrity. Every registered model carries a checksum; a mismatch makes the engine refuse to load rather than run quietly. Models are replaceable plugins and are never silently swapped — the version that produced a result is written into it.
• Web surface. Strict Content-Security-Policy with nonce-authorised scripts, nosniff, no framing, no referrer leakage, HSTS when served over TLS, per-principal rate limiting.
• Failure behaviour. No stack traces and no secrets in any response, ever; an internal error returns a problem document with a correlation id and nothing else.
• Evidence. Every administrative action and authentication event is written to an audit log with the actor, the outcome and the request id.

---

12. Licensing, packaging and the road ahead

Packaging. A single deployable unit: the application image (which builds the console and the server in one pass), the compose files, the neural sidecar image as an optional overlay, and the documentation set — an acceptance contract, an engineering design, a glossary, the calibration record of every measurement the engines rest on, an architecture decision record for the detector seam, the research-coverage matrix, and the deployment guide including Coolify.

Licensing. The application is proprietary. The neural model weights are not included: they are distributed by their authors under their own terms, and the sidecar's fetch script names where each one comes from and pins its digest. The built-in engines are part of the product and carry no third-party weights at all, which is why the product runs from a clean install with no downloads.

Commercial model. Per-deployment subscription through the Sensifai Marketplace, with the optional advisory layer licensed separately. The deployment reports usage through a local outbox that is flushed on a schedule, so a marketplace outage delays reporting rather than losing it — and the outbox is off entirely for a deployment with no marketplace relationship.

Roadmap, in the order it is worth doing.

1. Measure the operating point. A labelled evaluation harness producing EER, t-DCF and calibration curves, published with the corpus named. This is the single most valuable increment the product can make, because it converts "the engine scored 0.93" into "at this operating point, this is what the error rate is".
2. Face-region analysis for video. The neural path already localises faces; per-region scoring and a NO_FACE_DETECTED reason with an artefact-level explanation is the natural next step.
3. Operational ingestion. Watch folders, S3-compatible object storage, and a documented adapter interface, so the product can sit in a workflow rather than beside it.
4. Interoperability. A STIX 2.1 and CACAO exporter over the existing reason-code vocabulary, for organisations that push findings into a threat-intelligence platform.
5. Batch and throughput. A batch submission endpoint and batched GPU inference for customers processing archives rather than single recordings.

---