LCN Wafer Inspection

Appearance

Visualization & Operator Feedback in Industrial Vision Systems ​

This topic fits directly under your roadmap’s Vision, Imaging & Inspection Systems, specifically “overlay and review visualization” and “real-time image/result presentation” .

PART 1 — WHY VISUALIZATION IS CRITICAL ​

In an industrial vision system, visualization is not just “showing an image.” It is how the machine explains its decisions to the operator.

A vision algorithm may say:

Wafer die A17 failed because defect area exceeded threshold.

But the operator needs to see:

  • where the defect is
  • how large it is
  • whether the detection looks reasonable
  • whether the machine is behaving consistently
  • whether the result matches what the physical product looks like

In a wafer inspection machine, the operator may see:

text
Raw wafer image
+ red defect marker
+ ROI boundary
+ measurement line
+ PASS / FAIL status
+ confidence / score / threshold

The visualization becomes the human-facing evidence behind the inspection result.

A strong industrial UI answers:

“Why did the machine make this decision?”

A weak UI only says:

“Fail.”

That difference matters a lot.

If operators cannot understand what the system is doing, they stop trusting it. When trust drops, operators may start ignoring alarms, manually overriding results, restarting jobs unnecessarily, or escalating issues that are actually visualization problems.

A real production problem may look like this:

text
Operator sees:       PASS image
Machine recorded:    FAIL result
MES receives:        FAIL
Engineer log says:   Defect found at X=1203, Y=884

Even if the inspection algorithm is correct, the system feels broken because the operator UI and machine result are inconsistent.

In industrial systems, operator trust is part of system correctness.

PART 2 — IMAGE DISPLAY PIPELINE ​

The visualization pipeline is connected to acquisition and processing, but it must not control their timing.

The core pipeline is:

text
Camera / Acquisition
        ↓
Image Buffering
        ↓
Processing / Inspection
        ↓
Inspection Result
        ↓
Visualization
        ↓
Operator UI

A better architecture view:

text
+------------------+      +------------------+      +---------------------+
|  Acquisition     | ---> |  Processing      | ---> | Inspection Result   |
|  Camera/Trigger  |      |  Detection/ROI   |      | Defects/Measurements|
+------------------+      +------------------+      +---------------------+
                                                                |
                                                                v
                                                     +----------------------+
                                                     | Visualization Stream |
                                                     | Image + Metadata     |
                                                     | Result + OverlayData |
                                                     +----------------------+
                                                                |
                                                                v
                                                     +----------------------+
                                                     | Operator UI          |
                                                     | Display / Review     |
                                                     +----------------------+

The important rule:

The UI consumes inspection data. It must not become part of the timing-critical inspection path.

Bad design:

text
Processing waits for UI to draw image
Processing waits for operator screen to finish rendering
Processing stores image references because UI still uses them

Good design:

text
Processing publishes image/result package
UI consumes latest available display data
UI may skip frames
Processing continues independently

In real machines, inspection throughput is more important than displaying every frame. The UI should be responsive and truthful, but it should not slow down acquisition or processing.

PART 3 — OVERLAYS & INSPECTION VISUALIZATION ​

Overlays are the visual language of inspection.

Typical overlays include:

text
+ defect markers
+ bounding boxes
+ ROI boundaries
+ measurement lines
+ pass/fail coloring
+ alignment/fiducial markers
+ threshold indicators
+ classification labels
+ confidence score

Example:

text
+------------------------------------------------+
|                                                |
|        Wafer / Product Image                   |
|                                                |
|             [ROI-12]                           |
|          +------------+                        |
|          |     x      |  red x = defect        |
|          |            |                        |
|          +------------+                        |
|                                                |
| Measurement: width = 12.4 µm                   |
| Limit: <= 10.0 µm                              |
| Result: FAIL                                   |
+------------------------------------------------+

The overlay must be tied to the correct image and coordinate system.

This sounds simple, but in production it is a common source of serious bugs.

The system may have several coordinate spaces:

text
Camera pixel coordinates
Image buffer coordinates
Processed image coordinates
Downscaled display coordinates
Machine stage coordinates
Wafer/die coordinates
Operator screen coordinates

A defect detected at pixel (3200, 1800) in the raw image may need to be displayed on a downscaled image at (800, 450). If the image was cropped, rotated, flipped, registered, or corrected, the overlay must follow the same transform.

A strong design treats overlays as structured data, not as random drawing code.

text
InspectionResult
  - ImageId
  - CameraId
  - FrameId
  - Timestamp
  - CoordinateSystem
  - Defects[]
  - Measurements[]
  - ROIs[]
  - TransformMetadata

Then the UI can render overlays correctly.

You should separate these three concepts:

text
Raw image
  The original image from acquisition.

Processed image
  Image after filtering, thresholding, normalization, or transformation.

Annotated image
  Display image with overlays drawn on top.

Do not confuse them.

A common mistake is storing only an annotated image. That makes later debugging difficult because engineers cannot tell whether the defect came from the original image, the processing stage, or the overlay renderer.

PART 4 — REAL-TIME VS REVIEW VISUALIZATION ​

Industrial vision systems usually need two visualization modes.

Real-time mode ​

Real-time display is for monitoring the machine while it runs.

Its goals are:

text
- show current machine behavior
- show recent images quickly
- give operator confidence that inspection is active
- show pass/fail status
- show obvious defects or alarms

Real-time mode may skip frames.

That is acceptable.

If the camera captures 100 images per second, the UI does not need to display 100 full-resolution images per second. It may display 10 or 15 representative frames per second, or only the latest frame.

Real-time display prioritizes:

text
responsiveness > completeness
freshness       > full resolution
clarity         > excessive detail

Review mode ​

Review mode is different.

It is used after inspection, during investigation, defect review, quality decision, or engineering diagnosis.

Review mode prioritizes:

text
full resolution
accurate overlays
zoom and pan
comparison views
image history
measurement details
raw vs processed view
pass vs fail comparison

Review mode may load stored images or cached inspection records. It does not need to keep up with live acquisition speed.

A good architecture does not force real-time display and review display to use the same performance model.

text
Real-time mode:
  latest image, sampled frames, downscaled display, low latency

Review mode:
  selected image, full resolution, detailed overlays, rich interaction

Trying to make one screen do both usually creates a messy UI.

PART 5 — PERFORMANCE & DATA FLOW CONSTRAINTS ​

Images are large. Industrial cameras may produce high-resolution frames continuously. A naive UI can destroy system performance.

Common problems:

text
- UI tries to display every image
- UI keeps references to old images
- image buffers cannot be released
- memory grows over time
- GC pressure increases
- processing waits for rendering
- operator screen freezes

A robust design uses a consumer model.

text
+------------------------+
| Processing Pipeline    |
| Acquisition/Inspection |
+------------------------+
            |
            v
+------------------------+
| Result Stream          |
| ImageRef + Metadata    |
| InspectionResult       |
+------------------------+
            |
            v
+------------------------+
| UI Consumer            |
| Sampling / Downscale   |
| Overlay Rendering      |
+------------------------+
            |
            v
+------------------------+
| Operator Display       |
+------------------------+

The UI should usually receive a display-friendly package:

text
DisplayFrame
  - DisplayImage
  - ImageId
  - FrameId
  - Timestamp
  - CameraId
  - InspectionResult
  - OverlayData
  - DisplayScale

Important strategies:

text
Frame sampling
  Display only latest frame or 1 in N frames.

Downscaling
  Use smaller images for live display.

Asynchronous updates
  UI updates independently from processing.

Bounded queues
  Prevent unlimited image buildup.

Latest-frame policy
  If UI is behind, drop old display frames.

Clear ownership
  UI must not hold processing buffers forever.

The most important principle:

The UI should degrade gracefully under load.

For example:

text
Normal:
  display every 5th frame

Under load:
  display latest available frame only

Severe load:
  show status and thumbnail, stop live full image display

Never:
  block acquisition or inspection because UI is slow

PART 6 — OPERATOR FEEDBACK & INTERACTION ​

Operator feedback is a controlled loop.

text
+------------------+        +------------------+
| Machine System   | -----> | Operator UI      |
| Result/Alarm     |        | Image/Overlay    |
+------------------+        +------------------+
          ^                         |
          |                         v
+------------------+        +------------------+
| Workflow System  | <----- | Operator Action  |
| Continue/Review  |        | Ack/Override/Edit|
+------------------+        +------------------+

Operators may:

text
- acknowledge alarms
- review defects
- accept or reject parts
- override inspection decisions
- adjust allowed parameters
- navigate image history
- switch camera views
- compare raw and processed images
- request reinspection

But interaction must be controlled.

In industrial systems, not every operator should be allowed to change thresholds, override fails, or continue after certain alarms.

A strong design includes:

text
Role-based permissions
  Operator, engineer, maintenance, admin.

State-aware commands
  Some actions only allowed in review mode, manual mode, or stopped state.

Traceable actions
  Who did what, when, and why.

Clear confirmation
  Especially for overrides, recipe changes, or bypass-like actions.

Consistent result state
  UI action must update workflow state explicitly, not silently.

Example:

text
Operator clicks "Accept Override"
        ↓
UI sends OverrideInspectionDecision command
        ↓
Workflow validates:
  - user role
  - machine state
  - result state
  - reason code
        ↓
System records action
        ↓
Result becomes "AcceptedByOverride"
        ↓
UI refreshes displayed state

Do not let the UI directly mutate inspection data.

The UI should request an action. The workflow/domain layer decides whether the action is allowed.

PART 7 — REAL-WORLD FAILURE SCENARIOS ​

1. Overlay misaligned due to coordinate mismatch ​

What it looks like:

text
Defect marker appears next to the defect, not on it.
Measurement line is shifted.
ROI box does not match the inspected area.

Why it happens:

text
- image was downscaled but overlay was not scaled
- image was cropped but coordinates stayed raw
- camera orientation changed
- processed image coordinates were displayed on raw image
- registration transform was not applied

How experienced engineers diagnose it:

text
- compare raw image size vs display image size
- log coordinate system names
- inspect image transform metadata
- display calibration grid or known fiducials
- test with synthetic image and known coordinates

How to handle it:

text
- make coordinate system explicit
- store transform metadata with result
- centralize coordinate conversion
- never let each UI screen invent its own math

2. UI shows stale image while machine uses new data ​

What it looks like:

text
Operator sees image from previous wafer.
Current result belongs to new wafer.
Status says "Running", but image appears unchanged.

Why it happens:

text
- UI update queue is delayed
- result and image are published separately
- image cache reused wrong key
- UI binds latest result to old image
- multiple streams are not correlated by frame ID

How to diagnose:

text
- display ImageId, FrameId, WaferId, CameraId
- compare UI timestamp with processing timestamp
- check stream correlation
- inspect queue latency

How to handle it:

text
- package image + result + metadata together
- use immutable display snapshots
- include visible diagnostic identifiers in engineering mode
- discard stale frames when UI falls behind

3. UI freezes due to large image handling ​

What it looks like:

text
Live image stops updating.
Buttons respond slowly.
Operator thinks machine is stuck.
Processing may still be running.

Why it happens:

text
- full-resolution images rendered too often
- image conversion done on UI thread
- UI keeps many image references
- overlay drawing is too expensive
- no frame sampling

How to diagnose:

text
- check UI thread CPU usage
- measure render/update time
- inspect memory growth
- count queued display frames
- test with live display disabled

How to handle it:

text
- downscale before UI display
- render latest frame only
- move conversion off UI thread
- use bounded display queues
- separate live display from review display

4. Operator sees PASS but system recorded FAIL ​

What it looks like:

text
UI shows green PASS.
Report says FAIL.
Factory system receives FAIL.
Operator loses trust immediately.

Why it happens:

text
- UI status updated from preliminary result
- final result changed after later inspection step
- UI displayed die-level pass while wafer-level result failed
- old result state not cleared
- race condition between result updates

How to diagnose:

text
- inspect result lifecycle
- identify preliminary vs final states
- check event ordering
- correlate UI display state with persisted result state

How to handle it:

text
- model result states explicitly
- distinguish Preliminary, Final, Overridden, Invalidated
- update UI from authoritative result state
- avoid duplicated pass/fail logic in UI

5. Visualization hides important defect due to scaling ​

What it looks like:

text
Small defect is visible in full resolution but disappears in live view.
Operator says machine is falsely failing.
Engineer zooms in and sees actual defect.

Why it happens:

text
- display image is downscaled
- defect marker too small
- contrast lost in thumbnail
- overlay hidden by color or opacity
- small defects not emphasized

How to diagnose:

text
- compare live display vs full-resolution review
- test known small defects
- inspect downscale algorithm and marker size

How to handle it:

text
- show defect marker independent of pixel visibility
- provide zoom-to-defect action
- use review mode for final judgment
- warn when live image is display-scaled

6. UI slows down processing pipeline ​

What it looks like:

text
Inspection throughput drops when live image view is open.
Machine runs faster when UI screen is minimized or hidden.

Why it happens:

text
- processing publishes images synchronously to UI
- UI holds buffer ownership too long
- rendering shares locks with processing
- event handlers run expensive logic

How to diagnose:

text
- benchmark with UI enabled/disabled
- measure queue latency
- check locks around image buffers
- profile event handlers

How to handle it:

text
- decouple pipeline from UI
- use non-blocking publish/subscribe
- copy or transform only display data
- enforce bounded queues and drop policies

7. Multiple cameras or streams incorrectly displayed ​

What it looks like:

text
Camera A image shown with Camera B overlay.
Top camera result appears in side camera panel.
Operator reviews wrong image.

Why it happens:

text
- stream identity missing
- camera ID not part of display key
- result stream merged incorrectly
- UI assumes single camera
- asynchronous updates arrive out of order

How to diagnose:

text
- display CameraId and FrameId in debug overlay
- verify stream routing
- inspect correlation metadata
- replay recorded multi-camera sequence

How to handle it:

text
- include CameraId everywhere
- use stream-specific consumers
- validate image/result pairing
- design multi-camera UI intentionally

PART 8 — SOFTWARE DESIGN IMPLICATIONS ​

A good architecture looks like this:

text
+-----------------------------+
| Acquisition / Camera Layer  |
+-----------------------------+
              |
              v
+-----------------------------+
| Image Buffer / Stream Layer |
+-----------------------------+
              |
              v
+-----------------------------+
| Processing / Inspection     |
+-----------------------------+
              |
              v
+-----------------------------+
| Result Stream               |
| ImageRef                    |
| Metadata                    |
| InspectionResult            |
| OverlayData                 |
+-----------------------------+
              |
              v
+-----------------------------+
| Visualization Service       |
| Sampling                    |
| Scaling                     |
| Overlay Projection          |
| Display Snapshot Creation   |
+-----------------------------+
              |
              v
+-----------------------------+
| Operator UI                 |
| Live View / Review View     |
+-----------------------------+

The key design implications are:

1. Visualization must be decoupled from the core pipeline ​

The processing pipeline owns correctness and throughput.

The UI owns presentation and interaction.

They should communicate through explicit events, streams, or snapshots.

Bad:

text
Inspection code directly updates UI controls.
UI directly reads internal processing buffers.
Processing waits for rendering.

Good:

text
Inspection publishes result package.
Visualization service creates display snapshot.
UI renders snapshot.

2. Image + metadata consistency is non-negotiable ​

Every displayable image should have identity.

text
ImageId
FrameId
CameraId
Timestamp
RecipeId
WaferId / PartId
InspectionStepId
CoordinateSystem

Without metadata, the UI cannot prove what it is showing.

3. Overlay rendering must be coordinate-aware ​

Overlay data should not be random screen coordinates.

It should be expressed in a known coordinate system and transformed intentionally.

text
Inspection coordinate
        ↓
Image coordinate
        ↓
Display coordinate

The conversion should be centralized and testable.

4. Real-time and review modes should be separated ​

Do not force live display to behave like engineering review.

text
Live:
  fast, sampled, downscaled, latest state

Review:
  accurate, full resolution, detailed, navigable

This separation prevents many performance and usability problems.

5. Operator actions must be traceable ​

Any important operator action should be treated as a domain command, not just a button click.

Examples:

text
AcknowledgeAlarm
AcceptDefectReview
OverrideInspectionResult
RequestReinspection
ChangeDisplayMode
AdjustReviewThreshold

For sensitive actions, the system should record:

text
who
when
what
previous value
new value
reason
machine state
related image/result ID

Bad vs good approach ​

Bad approach:

text
UI directly consumes camera frames.
UI draws overlays with ad-hoc coordinate math.
UI stores all images in memory.
UI blocks processing when rendering is slow.
UI computes pass/fail display separately.
Live and review modes are mixed.
Operator overrides directly change result fields.

Good approach:

text
UI subscribes to display snapshots.
Processing publishes image + metadata + result.
Overlay rendering uses explicit coordinate transforms.
Live display samples/downscales images.
Review mode loads full-resolution data.
Result state is authoritative and centralized.
Operator actions go through workflow/domain commands.

PART 9 — INTERVIEW / REAL-WORLD TALKING POINTS ​

A strong interview answer could sound like this:

In an industrial vision system, visualization is part of the machine’s correctness story, not just a UI feature. The operator needs to see the image, the inspection result, and the reason behind the decision in a synchronized way. That means the UI must display the correct image, with the correct overlays, from the correct inspection result, using the correct coordinate transform.

Then continue:

Architecturally, I would not let the UI sit in the acquisition or processing path. The processing pipeline should publish image/result packages with metadata, and the UI should consume display snapshots asynchronously. For live display, I would allow frame sampling and downscaling, because responsiveness matters more than showing every frame. For review mode, I would support full-resolution images, zoom, comparison, and detailed overlays.

And finish with:

The biggest risks are stale images, misaligned overlays, UI freezes, inconsistent pass/fail display, and memory growth from image retention. To avoid those, I would use explicit image IDs, frame IDs, timestamps, camera IDs, coordinate-system metadata, bounded UI queues, clear ownership of image buffers, and traceable operator actions.

Common mistakes software engineers make when entering this domain:

text
- treating image display like normal UI rendering
- assuming the UI can display every frame
- ignoring coordinate systems
- mixing raw, processed, and annotated images
- letting UI logic decide pass/fail state
- holding image buffers too long
- not separating live display from review mode
- forgetting that operator trust depends on visual consistency

What strong engineers understand:

text
Visualization is evidence.
Overlays must be accurate.
Image/result pairing must be explicit.
The UI must not slow the machine.
Live display and review display are different products.
Operator actions are part of the machine workflow.
Trust is built through clarity, consistency, and traceability.

The simplest architectural principle is:

The inspection pipeline produces truth. The visualization layer presents truth. The operator UI must never invent, delay, corrupt, or obscure that truth.

Docs-first project memory for AI-assisted implementation.

LCN Wafer Inspection Prototype