Room Type
Kitchen, bedroom, bathroom, living room, exterior, garage, etc.
Understanding Vision Analysis
Vision analysis automatically examines property photos, enabling powerful visual search that goes beyond listing descriptions. Instead of relying solely on what agents write in MLS descriptions, buyers can search based on what they actually see in photos.
The Core Problem
Section titled “The Core Problem”Traditional property search has a fundamental limitation: it can only find what’s written in the listing description. If an agent forgets to mention “granite countertops” or “mountain views,” that property won’t show up in searches—even though those features are clearly visible in the photos.
Vision analysis solves this by examining the photos themselves and automatically detecting features, styles, materials, and room types.
What It Does
Section titled “What It Does”Every property photo is automatically analyzed to identify:
Features
Countertops, appliances, flooring, fixtures, built-ins
Materials
Granite, hardwood, tile, stainless steel, quartz, marble
Style & Condition
Modern, traditional, farmhouse, craftsman, updated, original
How It Works
Section titled “How It Works”The process happens in several stages:
1. Photo Ingestion
Section titled “1. Photo Ingestion”When a property syncs from MLS, all photos are downloaded and stored locally. Each photo gets a unique ID and is linked to the listing.
2. Visual Analysis
Section titled “2. Visual Analysis”Each photo is processed through two different AI models:
MiniCPM-V generates natural language descriptions:
“Modern kitchen with white cabinets, granite countertops, stainless steel appliances, and pendant lighting over a large island. Hardwood floors and subway tile backsplash.”
SigLIP generates visual embeddings (mathematical representations of what the photo looks like). These embeddings enable “find me photos that look like this” searches.
3. Feature Extraction
Section titled “3. Feature Extraction”The natural language descriptions are parsed to extract structured features:
| Category | Extracted Features |
|---|---|
| Room Type | kitchen, bedroom, bathroom, living_room, exterior, garage |
| Materials | granite, hardwood, tile, stainless_steel, quartz, marble |
| Features | island, pantry, fireplace, pool, deck, walk_in_closet |
| Style | modern, traditional, farmhouse, craftsman, contemporary |
| Appliances | gas_range, double_oven, dishwasher, refrigerator |
4. Indexing
Section titled “4. Indexing”Both the natural language descriptions and visual embeddings are indexed for fast search:
- Text search uses full-text indexing on descriptions
- Visual search uses vector similarity on embeddings
- Multi-modal search combines both for best results
Search Capabilities
Section titled “Search Capabilities”Vision analysis enables three types of search:
Description Search
Section titled “Description Search”Search using automatically generated photo descriptions. The system searches the detailed text generated for each photo.
Example query: “granite countertops and stainless steel appliances”
This finds all photos whose descriptions mention those features—even if the listing description doesn’t.
Visual Similarity Search
Section titled “Visual Similarity Search”Find photos that look similar using visual embeddings. This searches directly against what’s in the image, not the text describing it.
Example query: “modern white kitchen”
The system finds kitchens that visually resemble modern white kitchens—similar color palettes, layout, style—regardless of how they’re described.
Multi-Modal Search
Section titled “Multi-Modal Search”Combines text and image search for the best results. Returns properties ranked by both textual relevance and visual similarity.
Example query: “open concept living room with vaulted ceilings”
Results are scored on both:
- How well the description matches the query
- How visually similar the photos are to typical open-concept vaulted living rooms
Practical Applications
Section titled “Practical Applications”For Buyers
Section titled “For Buyers”Vision analysis means you can search for what you actually want to see, not just what’s written:
- “Show me kitchens with white cabinets” — finds them even if description says “light cabinetry”
- “Find properties with mountain views” — detects views in photos even if not mentioned
- “Houses with updated bathrooms” — identifies modern fixtures and finishes visually
You’re no longer limited by how thoroughly agents wrote their descriptions.
For Agents
Section titled “For Agents”Vision analysis helps your listings get discovered:
- Your photos do the talking—features don’t need to be in the description to be searchable
- Buyers can find your listing through visual searches
- Properties with great photos perform better in search results
This rewards good photography and complete photo galleries.
Photo Intelligence Features
Section titled “Photo Intelligence Features”Room Detection
Section titled “Room Detection”Photos are automatically categorized by room type, making it easy to:
- View all kitchen photos across listings
- Compare backyard and pool images
- Find master bedroom suites
- See garage and workshop spaces
Feature Tagging
Section titled “Feature Tagging”Each photo gets detailed auto-generated tags:
| Room Type | Example Tags |
|---|---|
| Kitchen | island, pantry, breakfast bar, gas range, subway tile backsplash |
| Bathroom | soaking tub, walk-in shower, double vanity, heated floors |
| Living Room | vaulted ceilings, fireplace, built-in shelving, hardwood floors |
| Exterior | covered patio, pool, mountain views, landscaped yard, 3-car garage |
These tags enable precise filtering and searching.
For Agents: Optimizing Your Listings
Section titled “For Agents: Optimizing Your Listings”Photo Quality Matters
Section titled “Photo Quality Matters”Vision analysis is only as good as your photos. To maximize discoverability:
- Take high-quality photos — Well-lit, clear images get better analysis
- Show key features — Include close-ups of granite, hardwood, fixtures
- Capture room context — Wide shots show layout and style
- Highlight unique features — Pool, views, custom built-ins deserve their own photos
Photo Order Matters
Section titled “Photo Order Matters”The system analyzes all photos, but prominent features in early photos get weighted higher. Structure your photo gallery:
- Hero photo — Best feature or exterior curb appeal
- Living spaces — Great room, kitchen, dining
- Bedrooms — Master suite first
- Bathrooms — Master bath, then guest baths
- Special features — Pool, views, bonus rooms
- Exterior/yard — Backyard, landscaping, garage
Understanding Engagement
Section titled “Understanding Engagement”Vision analysis tracks which features attract attention:
- Dwell time — How long buyers view each photo
- Zoom interactions — Photos buyers zoom into for detail
- Feature patterns — Which visual features correlate with engagement
Use this data to understand what buyers value and adjust future listings accordingly.
Privacy & Security
Section titled “Privacy & Security”Vision analysis respects privacy:
- Only MLS property listing photos are analyzed
- No personal photos or documents are processed
- All processing follows MLS data usage guidelines
- Descriptions are stored locally, not sent to external services after processing
Photos are analyzed once during initial sync. Results are cached for fast search performance.
Limitations
Section titled “Limitations”Vision analysis is powerful but not perfect:
What It Does Well
Section titled “What It Does Well”- Identifying obvious features (pools, fireplaces, granite counters)
- Detecting room types (kitchen vs. bathroom vs. bedroom)
- Recognizing materials and finishes
- Understanding general style (modern vs. traditional)
What It Struggles With
Section titled “What It Struggles With”- Subtle details in low-quality photos
- Features obscured by furniture or staging
- Exact measurements (can’t measure square footage from photos)
- Condition assessment (can’t definitively say “newly renovated”)
- Features not visible in photos (buried utilities, structural elements)
The system provides confidence scores for detected features. Low-confidence detections are filtered out to maintain accuracy.
Technical Details
Section titled “Technical Details”Model Information
Section titled “Model Information”| Model | Purpose | Provider |
|---|---|---|
| MiniCPM-V | Natural language photo descriptions | OpenBMB |
| SigLIP | Visual embeddings for similarity search | Google Research |
Both models run locally via Ollama—no photos are sent to external APIs.
Processing Time
Section titled “Processing Time”- Per photo: ~2-3 seconds on GPU, ~10-15 seconds on CPU
- Full property (20 photos): ~1 minute on GPU, ~5 minutes on CPU
- Batch indexing: Parallelized across available cores
Storage Requirements
Section titled “Storage Requirements”Each analyzed photo requires:
- Original image: ~500KB - 2MB (JPEG)
- Description text: ~200-500 bytes
- Visual embedding: 512 floats × 4 bytes = 2KB
For 10,000 properties with 20 photos each:
- Total photos: 200,000
- Storage needed: ~200GB images + ~400MB embeddings
Future Enhancements
Section titled “Future Enhancements”We’re exploring additional vision capabilities:
- Floor plan analysis — Automatically detect room layouts
- Virtual staging — Show how empty rooms could look furnished
- Style matching — “Find homes with similar style to this one”
- Feature comparison — Side-by-side visual comparisons
- Trend detection — “Most popular kitchen styles this year”
Related Features
Section titled “Related Features”- Vision API Reference — Technical API documentation
- Semantic Search — Natural language property search
- Understanding Scribe — How Scribe uses photo engagement data