Catalog your whole bookshelf from one photo

The idea

Point your phone at a bookshelf, take one photo, and get back a list of every book on it. The app segments the image into individual spines, runs text recognition on each one rotated upright, matches the extracted title and author against a book metadata database, and hands you a confirmation list to swipe through. What you keep is a plain catalog of your own library — searchable, exportable, stored on the device.

Why build this

People who own a lot of books mostly don't know what they own. The failure modes are concrete: buying a second copy, lending a book and forgetting to whom, standing in a used bookstore unable to remember whether you already have the sequel. The existing answer is Goodreads or LibraryThing, which means scanning barcodes one at a time — twenty seconds per book, so nobody finishes a shelf, let alone a room.

The reason to build it now is that the hard part got cheap. Rotated-text recognition used to be the wall; Apple's Vision framework and Google's ML Kit both handle arbitrary text orientation on-device at usable accuracy, and small segmentation models run in tens of milliseconds on a modern phone. Open Library exposes a free search API over roughly forty million editions. The whole pipeline now fits on the handset, which also means the photos never have to leave it.

Stack sketch

• React Native via Expo, with expo-camera for capture and a dev build so native vision modules link in
• Spine segmentation: a small YOLO-style detector trained on shelf photos, exported to ONNX and run through onnxruntime-react-native — or, for v1, a classical approach using vertical edge density from OpenCV to slice the image into columns
• Text recognition: Apple Vision (VNRecognizeTextRequest) on iOS, ML Kit Text Recognition v2 on Android, both configured for rotated text
• Metadata matching: Open Library Search API as primary, with fuzzy scoring over title plus author using a trigram similarity in JS; ISBNdb as a paid fallback for the misses
• Local store: expo-sqlite with an FTS5 table over title, author, and series so search stays instant offline
• Covers: Open Library Covers API, cached to the filesystem

Scope for v1

In: one photo at a time, spine slicing, OCR, top-three candidate matches per spine, a swipe-to-confirm review screen, a local catalog with full-text search, CSV and JSON export.

Out: accounts, cloud sync, sharing, recommendations, lending, reading progress, barcode scanning as a fallback path, and any attempt to detect books lying flat or stacked horizontally. One shelf, spines out, decent light.

Where it could go

The obvious next layer is availability. Once the catalog exists, a "do I already have this" lookup while you're standing in a shop is one screen away, and the inverse — checking whether your local library holds the book you almost bought — is a WorldCat or library-consortium API call. That turns a static inventory into something you consult weekly rather than once.

After that: lending. A book you photographed is a book you can mark as out with someone, with a reminder. And a household or shared-house mode, where several people's shelves merge into one searchable catalog, is the first thing that justifies a server at all. Keep that optional — the local-only version should stay fully useful for people who never sign in.

Watch out for

The accuracy floor is set by the worst spines, not the average ones: worn cloth bindings, decorative type, and books shorter than the OCR's minimum text height will simply not resolve, and users will judge the whole app on those. Design the review screen so unmatched spines are cheap to fix by hand rather than treating them as errors, and be honest in onboarding that a shelf photo gets you eighty percent of the way, not all of it.