PERSONAL PROJECT · DATA TOOLING
Refinery
Make collected information useful.
A command-line project for collecting, organizing, and searching web data.
Conceptual illustration, not a product screenshot
The question
Collecting information is only a first step. Refinery explores how retrieved web data can be organized, searched, and used by other tools without losing the context around how it was collected.
The approach
The project combines a crawler with SQLite storage and full-text search. An MCP interface, local REST API, and documented GraphQL subset provide different ways to work with the collected information.
What I explored
The design pays attention to crawl rules, request boundaries, and retaining evidence about retrieved information. The development workflow separated planning, implementation, and review, connecting data-integration thinking with practical AI tooling.
Personal command-line tool. Designed for local use.