Get in touch

Smart Gate, Two-Factor Vehicle Access Control

AboutAugust 2026

09· AI EngineeringUndergraduate thesis

A campus-gate system that checks a motorcycle at exit against whoever actually rode it in, reading the plate and recognizing the rider's face, so a borrowed or stolen bike is refused the moment a different person tries to ride it out.

computer-vision
The whole system, end to endFig. 3.1
Three datasets feed two trained models (a face model and a plate detector), which feed four runtime systems (object detection, face recognition, plate reading, and the matcher that decides). A model only leaves the training loop once its evaluation passes; a bad result routes straight back to retraining. Everything below is one of these boxes opened up.
What the camera actually gives youFig. 3.4
Left, a usable capture: face and plate both legible. Right, the same gate a few hours later, with the plate cropped by the frame edge and the rider's face behind a full-face helmet. This is passive CCTV, not a purpose-built gate camera, and plates land at roughly 60-120 pixels wide. Every design decision further down exists because of this picture.
Dual-model detection pipelineFig. 3.8
Every video frame runs through two detection models at the same time (one for riders and motorcycles, one for plates), so the wait on one model is spent running the other instead of adding to the total delay, which matters for a live camera feed. Running two models rather than one is also deliberate redundancy: if either misses on a given frame, the other still supplies evidence.
When the plate isn't found at allFig. 3.7
Detection rarely fails on everything at once. If the plate is missed but the rider is found, the system estimates where the plate should be from the motorcycle's position, and vice versa. A capture only drops out of the pipeline entirely when neither piece of evidence survives, so a partial detection still reaches the matcher instead of being thrown away.
Plates are never square to the cameraFig. 3.9
A detector returns an upright rectangle, but a plate photographed from the side sits at an angle inside it. Fitting a rotated box to the plate recovers that angle, which is what lets the next step flatten the plate before reading it. Left, a near-square case; right, a plate rotated roughly fifteen degrees, read correctly as AG6544VBG at 0.97 confidence.
Flattening the plate before reading itFig. 3.15
If the rotated outline from the previous step is available, the plate is un-warped in perspective as though the camera were square to it. If it isn't, the system falls back to simply de-skewing the crop. Either way the text reader receives a flat, upright plate rather than a trapezoid, which is worth several percentage points of readable characters on its own.
Multi-engine plate readingFig. 3.16
The main plate-reading engine goes first. Only if that reading fails Indonesia's plate format or comes back with low confidence do the two backup engines run and vote on the result, so all three engines running together is a fallback, not the normal case. Individually the three engines return a valid reading only 21%, 27% and 43% of the time; their combined vote reaches 69%, better than any of them alone.
Reading a plate in three pieces, not oneFig. 3.17
An Indonesian plate is a region prefix, up to four digits, and a letter suffix, and each zone has its own rules: a character that must be a digit can never be the letter O. Splitting the reading into three zones and correcting each against its own alphabet turns ambiguous characters into decidable ones, and lets the engines vote zone by zone rather than on the whole string at once.
Picking which video frames to readFig. 3.10
From a short clip, the system scores each tracked vehicle by how often it appears, how large it looks, and how far it moves, and only commits to the clear leader. Its 5 sharpest, clearest frames are what actually get read for the plate. This is what makes a blurred single frame survivable: no one frame has to be perfect.
Turning a face into a comparable fingerprintFig. 3.13
A rider's face is cropped, straightened, and run through a trained face-recognition model that turns it into a string of 512 numbers, a fingerprint that can be compared mathematically. The system then looks up the 3 closest-matching faces on file rather than betting everything on a single best guess.
The final exit decisionFig. 3.18
Every motorcycle still marked as inside is scored on both face match and plate match against the vehicle now leaving, and only the strongest, best-corroborated candidate is accepted. The plate gate has to pass first, and that ordering is the whole design: a face score, however high, can never let a pair through on its own. It is what keeps a weak face model from becoming a weak security system.
Where captures actually get lostFig. 4.9
Of 142 real CCTV captures, 134 produced a usable plate region and every one of those produced some text, but only 104 produced a reading complete enough to trust. Faces survived better, at 137. The narrow point is not finding things, it is reading them, which traces directly back to how few pixels a CCTV plate occupies.
What evidence each accepted match rested onFig. 4.10
47 of 71 entry-exit pairs were matched, and every accepted match carries a label saying why. 25 had both a full plate and an agreeing face. 19 were accepted on the plate alone, with the face absent or too weak to use. 3 rested on a partial plate rescued by a strong face. The 24 refusals were deliberate, each with a stated reason, not silent failures.
Why the face is supporting evidence, not the verdictFig. 4.11
Plate similarity (right) is bunched hard against 0.8-1.0 for every tier, because the plate gate is mandatory. Face similarity (left) is scattered all the way down to 0.1 for the plate-only tier. That spread is exactly the argument for the architecture: a system that had trusted the face would have accepted or refused those pairs on noise.
Entry, 08:00Use case 1
The swap gets caughtUse case 1
At 09:45, after Person B rode back in with the bike at 09:30, Person B tries to take it back out again. Same plate, but a different face than the one on record for this vehicle, so the exit is refused and both open sessions for it stay open.
Exit refused, 09:45Use case 1
Entry, 08:00Use case 2
Hand-off refusedUse case 2
At 08:30, Person B tries to ride the motorcycle straight back out. The plate reading still lines up closely, but the face match comes back well below what's needed to pass, so the exit is refused outright.
Exit refused, 08:30Use case 2
Entry, 08:00Use case 3
A legitimate exitUse case 3
At 08:15, Person A takes the same motorcycle back out. The face match comes back clearly above the threshold, so this is a clean match and the session closes normally.
Exit granted, 08:15Use case 3
Write-up

Gated housing complexes, campuses and paid parking in Indonesia mostly still verify vehicles the way they did thirty years ago: a guard glances at a plate, maybe at a ticket, and lifts the barrier. That check has a specific hole in it. It confirms the vehicle, never the person on it. Ride a borrowed or stolen motorcycle back out through the same gate and a plate-only system, human or automated, waves it straight through. Motorcycle theft in Indonesian parking areas works almost entirely through that hole.

Smart Gate closes it by checking two things at once when a bike leaves, and comparing both against whatever was recorded coming in: the plate, and the rider's face. When both agree the gate opens and the visit closes. When the plate matches but the face doesn't, the exit is refused, logged as a security event, and the motorcycle stays marked as still inside, exactly as if it never left. Nothing about the vehicle alone is ever sufficient.

The plate side is a pipeline, not a model. A detector finds the plate, a rotated outline recovers the angle it sits at, the crop is flattened in perspective, and only then is it read. The reading itself is split into an Indonesian plate's three zones (region prefix, digits, letter suffix) and each zone is corrected against its own alphabet, so a character that has to be a digit can never come back as the letter O. A primary text engine goes first, and two backup engines are called in only when that first reading fails the format check. Individually those three engines return a valid reading 21%, 27% and 43% of the time. Voting zone by zone across all of them reaches 69%, better than the best engine on its own, and swapping which engine leads changed the final match count by exactly zero.

The face side is a model I trained myself rather than a pretrained one, and its results are the most useful thing in the whole project. On its own curated test set it is close to perfect: an equal error rate of 1.59% and a ROC-AUC of 0.9967. Pointed at real gate CCTV, with riders it had never seen, helmets, motion and harsh sun, that same model degrades to a 28.7% equal error rate. The gap between those two numbers is the entire lesson. A benchmark score measured on faces drawn from the same pool as the training data says almost nothing about a model at a real gate.

So the architecture was built around distrusting the face rather than around the face being good. Every accepted pair has to clear a strict plate gate first (both sides read, all four core digits identical) before any similarity score is even considered. Only then does the face decide which tier the match gets labelled: both agreeing, plate-only, or a partial plate rescued by a strong face. A face score can never let a pair through by itself; it can only corroborate, or contradict. The closest published comparison, TRI-GATE, fuses its three signals into one number and opens the gate when that number clears a threshold. That is simpler, but a single number can't tell you afterwards which signal actually let a vehicle out. Here every decision carries an explicit evidence label, which is what makes the log worth anything in an investigation.

Evaluated on 142 real CCTV captures covering 71 entry-exit pairs, the system matched 47 of them (66.2%) with zero format or digit violations, and the plate detector reached 0.811 mAP@50. 19 of those 47 matches went through on the plate alone, with the face absent or too weak to use, which is only possible because the face was never load-bearing. The remaining 24 pairs were refused on purpose, each with a stated reason. That is the priority the whole thing is tuned to: refusing an unconvincing pair is cheap, and releasing the wrong vehicle is not.

66.2% only means something next to a comparison. TRI-GATE reports 97%, but on a dedicated gate camera with the vehicle stopped and the plate filling a good part of the frame. This ran on passive CCTV, plates 60-120 pixels wide, faces routinely behind helmets, and identities never seen during training. Most of the gap is task difficulty, and the rest is a reporting choice: TRI-GATE publishes a single combined accuracy, while this reports false-accept and false-reject rates in full, including the unflattering 28.7%.

A second input path takes short video clips instead of single frames, pooling evidence across frames before deciding, and adds a rider-continuity check aimed squarely at the theft case. A staged test (same motorcycle, different rider at exit) was correctly refused: the face scored 0.793, below the 0.85 needed to confirm but above the floor that would mark it as a different person entirely, so the pair was pushed to manual review rather than released. Three filmed hand-off scenarios ran the same way, with every mismatched rider caught, 12 out of 12, and no wrong rider ever let out. A clip that can't be resolved always fails closed, into the manual queue that already exists today.

The honest limits are all load-bearing, not footnotes. Image resolution is the real ceiling: 12 of 142 captures never produced a readable plate at all, and past roughly 66% the next gains come from moving or upgrading the camera, not from better code. Every threshold was tuned on one camera over one simulated day and would need revalidating elsewhere. Matching is greedy and order-dependent, so an early entry can claim an exit that suited a later one better. And the face model's open-set gap is unresolved; an earlier VGGFace2 backbone actually scored better on live footage (15.4% versus 28.7% equal error rate), which is documented in the thesis rather than quietly dropped.

The parts that make it deployable are the unglamorous ones. It runs on passive CCTV a complex already owns rather than imported gate hardware, which is the cost barrier keeping this class of system out of Indonesian residential and campus parking in the first place. It is built to Indonesian plate format rather than adapted from a Western one. Every decision, accepted or refused, leaves an auditable reason a guard or an investigator can read back. And when it is not sure, it hands the case to the human process already running the gate today, so it can be switched on beside existing staff instead of replacing them on day one.

Things to underline
  • Matched 47 of 71 entry-exit pairs from real CCTV (66.2%) with zero false accepts: no accepted pair violated plate format or digit agreement
  • Built the decision around a mandatory plate gate so a face score can never open the gate alone, which is why 19 of 47 matches still succeeded with the face absent or unusable
  • Reached 0.811 mAP@50 on the license-plate class, higher than the pretrained person (0.695) and motorcycle (0.486) classes in the same model
  • Raised usable plate readings from 43% (best single OCR engine) to 69% by voting three engines zone by zone; swapping the lead engine changed the final match count by zero
  • Labelled every accepted match with the evidence behind it rather than a single fused score, so an audit can tell which signal actually released a vehicle
  • Caught every staged rider-mismatch across three filmed hand-off scenarios (12 of 12), including a same-plate different-rider theft test refused at 0.793 face similarity
  • Face recognition fell from 1.59% equal error rate on its own test set to 28.7% on live CCTV, an open-set generalisation gap reported rather than hidden
  • Only 73.2% of captures produced a complete plate reading; 12 of 142 produced none at all, a limit set by CCTV resolution rather than by the algorithm
  • Every similarity threshold was tuned on one camera over one simulated day, so another site needs revalidation before the numbers carry over
  • Pairs entries to exits greedily in arrival order, so an early entry can consume an exit that would have suited a later one better
Built with
PythonYOLOv8ArcFaceOCR ensemble