Finding the Moment That Matters: How We Built a Semantic Search Engine for News Video
The best clip is never at the beginning.
Anyone who has hunted through hours of news footage knows the feeling. You can picture the moment — a minister conceding a point under pressure, an analyst calling the market three weeks before it turned, an anchor describing a scene as it unfolded live. You know it exists. You just don't know whether it's at the eleven-minute mark or the fifty-third, in this broadcast or the one before it. So you scrub. You drag the playhead back and forth, hunting for a needle in a haystack made of more hay.
That single, repeated act — scrubbing — quietly costs the media industry an enormous amount of time. We built a system to make it disappear.
This is the story of how we turned an archive of news video into something you can search the way you'd ask a colleague: "Find me the part where the Finance Minister talks about fiscal consolidation." And it finds it — not the start of the video, the moment.
The problem with video as a data format
Video is the richest medium of communication we have. It is also one of the hardest to search.
A text article can be scanned, keyword-matched, and ranked in milliseconds. A video cannot. Everything that makes a video valuable — the words spoken, the people speaking them, the context around them — is locked inside a stream of pixels and audio. To a traditional search engine, that content is invisible unless a human has already sat down and written it out by hand.
News makes this problem worse in every dimension. It is voluminous: a single broadcaster can produce hundreds of hours of footage a week. It is time-sensitive: the value of a clip can peak and decay within a single news cycle. And it is dense — political debates, earnings calls, press conferences, on-the-ground reporting, each one packed with dozens of individually quotable, individually searchable moments. The hard part was never storing all of it. The hard part is finding the right ten seconds inside it.
Our approach: every moment is a searchable document
The insight at the heart of our system is deceptively simple. A video is not one thing to search. It is thousands of things.
Every stretch of spoken content, every handover between speakers, every shift in topic is a distinct, retrievable unit of meaning. So instead of treating a 90-minute broadcast as a single opaque file, we break it into its constituent moments, teach the system what each moment is about, and store that understanding in a way that supports fast, natural-language retrieval.
Here is how the pipeline works, end to end.
Stage 1 — Ingestion and transcription
We ingest video from wherever it lives: YouTube channels, broadcaster RSS feeds, press-conference recordings, live broadcast archives. Each video is automatically transcribed and split by speaker using diarization — the process of working out who is talking at any given instant.
The output isn't a wall of text. It's a structured timeline: a sequence of labelled utterances, each pinned to a start and end timestamp measured in milliseconds. Every word spoken on screen becomes a precise, addressable point in time we can link straight back to.
Stage 2 — Speaker identification
Raw diarization only knows that Speaker A is different from Speaker B. We go a step further. For each unique speaker, we pull a short clip from their first appearance and pass it to a multimodal model that identifies who is actually talking — by name, role, and affiliation. That speaker map becomes part of every chunk's metadata.
This matters enormously at retrieval time. Someone searching for "the RBI Governor on the inflation outlook" shouldn't need to know which broadcast to open. The system already knows that the voice speaking at 14:32 belongs to the Governor — and can surface it directly.
Stage 3 — Semantic chunking
We don't slice transcripts at arbitrary intervals. We use a diarization-aware strategy that respects the natural seams of a conversation. A new chunk begins when the speaker changes, when a meaningful pause occurs, or when a sensible word-count threshold is reached. Context is deliberately overlapped at the boundaries, so no meaning is lost where one chunk ends and the next begins.
Each chunk becomes a self-contained document carrying full metadata: speaker name, video title and source URL, start and end timestamps, topic tags, and the plain text of what was said.
Stage 4 — Semantic embedding
Each chunk is then transformed into a high-dimensional vector embedding — a mathematical fingerprint of its meaning — using a state-of-the-art embedding model. This is the step that changes everything, because it lifts search out of the world of exact words and into the world of ideas.
Stage 5 — Storage in Qdrant on GCP
Every embedding and its metadata land in Qdrant, a purpose-built vector database hosted on Google Cloud Platform. Qdrant is optimised for approximate nearest-neighbour search at scale, which means that even as the index grows into the tens of thousands of chunks across thousands of videos, queries stay fast and results stay relevant. Each record holds the full payload — video URL, speaker, timestamps, source, original text — so a single match contains everything needed to jump straight to that moment.
Why keyword search was never going to be enough
To see why embeddings matter, picture a real query: "tech executive discussing job cuts."
Now picture the clip that should answer it — a CEO saying, "we are restructuring our workforce and reducing headcount by fifteen percent."
A keyword engine compares the words. It finds none in common. Not "tech," not "executive," not "job," not "cuts." Zero overlap, zero results — the clip is effectively invisible, even though it's a perfect answer.
A semantic engine compares the meaning. In vector space, "reducing headcount" sits right next to "job cuts," and "restructuring our workforce" sits right next to "discussing." The query and the clip land close together, and the system returns it with confidence — without a single shared word.
That gap — between matching words and matching meaning — is the entire reason this works. It's also why a system like this keeps finding things a keyword index would silently miss.
What retrieval looks like in practice
A user types a question in plain language. We embed that question with the same model used when the videos were indexed, so the query and the content live in one consistent semantic space. We then find the closest chunks in Qdrant.
Results come back with direct, timestamp-anchored links. If the matching moment sits at 22 minutes and 47 seconds inside a 90-minute broadcast, clicking the result takes you there — not to 00:00. For YouTube-hosted content the link encodes the timestamp natively; for hosted media, the player jumps straight to the segment.
The experience feels less like filtering a file cabinet and more like asking a well-briefed colleague who happens to have watched everything.
Results and scale with semantic search
This isn't a prototype on a single video. Since deploying the system, we've indexed a substantial, growing archive — and the numbers reflect what it's like to use at scale.
We've indexed over 1,200 news and broadcast videos across multiple channels and source types, representing more than 800 hours of total content. That footage has been broken into over 47,000 individually searchable chunks, each anchored to a precise timestamp and attributed to a named speaker.
Across test queries spanning political commentary, financial analysis, and breaking-news events, the system returns semantically relevant results in under 300 milliseconds — fast enough to sit inside a live editorial workflow rather than beside it. And in internal testing, the feedback was consistent: editors reported spending up to 80% less time finding specific clips compared with manual scrubbing. In a newsroom where speed is measured in seconds, that's not a convenience. It's a different way of working.
Why this matters beyond the newsroom
Faster clip-finding is the obvious win. It is not the most important one.
For journalism and fact-checking, every claim made on air becomes searchable and attributable. A statement a public figure made six months ago in a press conference can be pulled up in seconds and held against what they're saying today.
For archives and institutional memory, organisations that are sitting on decades of unsearchable footage get those archives back. No manual tagging, no re-cataloguing — the library simply becomes queryable.
For content production, producers building explainers, documentaries, or social cuts can search their entire library for the right soundbite instead of relying on memory and half-complete logs.
For analytics, because every result carries structured metadata — speaker, source, timestamp, topic — the search layer doubles as an analytics layer. You can track how often a topic comes up, who is driving the conversation, and how coverage shifts over time.
And none of this is specific to news. The pipeline doesn't care whether the speech it's indexing is a broadcast, a deposition, a lecture, a sales call, or twelve hours of security footage. If it's spoken on video, it can be made searchable.
That's the part worth sitting with. The problem we solved for newsrooms — "I know this moment exists, I just can't find it" — is the same problem faced by legal teams buried in deposition video, compliance teams reviewing recorded calls, universities sitting on lecture archives, and security operations staring at footage no one has time to watch. The mechanism is identical. Only the source changes.
The road ahead
We're continuing to expand the index and onboard new broadcast sources and historical archives. On the roadmap: multi-language support, entity-level search (treating a person, organisation, or place as a first-class searchable concept), and a summary layer that can synthesise everything said about a topic across many videos into a single coherent brief.
The vision is a media intelligence platform where no important moment goes unnoticed — and every moment that matters can be found in seconds.
If this sounds like your problem, let's talk
If you're sitting on an archive you can't search — a newsroom, a media library, a vault of recorded calls, a mountain of footage no one has the hours to log — that is exactly the kind of problem we build for. Turning vast, messy, unstructured video into something instantly searchable is what we do.
We'd genuinely like to hear what you're working with. Reach out through vigaet.com and let's find your moments wit semantic search.
Built with Gemini embedding models, Qdrant vector search, and a pipeline designed for the pace of news.
Comments