All Posts
Everything we've published, grouped by series. Forma has both English and Chinese editions; the other series are in English.
Forma: A Flexible Data Storage Engine for the AI Era
Why Forma?
Traditional databases weren't built for the AI era. When your AI Agent outputs 12 fields today and 30 fields tomorrow, waiting 3-7 days for DDL approval isn't an option.
Forma solves this with a modern take on the EAV pattern:
| Problem | Traditional DB | Forma |
|---|---|---|
| New field | ALTER TABLE (days) | JSON Schema update (seconds) |
| Schema change | Downtime required | Zero downtime |
| AI output | Manual adaptation | Direct JSON Schema mapping |
| N+1 queries | 101 round-trips | 1 round-trip |
| Historical data | Same table, same cost | Cold storage on S3 |
English Series
A three-part engineering series on building flexible, high-performance data storage for AI applications.
Series Introduction: From EAV to Zero-Dirty-Read Lakehouse
What Forma is and what problems it solves
Start here for an overview of the architecture and the three core problems behind it.
Part 1: Why EAV is the Most Underrated Data Model for AI
JSON Schema + hot table = AI-ready infrastructure
JSON Schema is the type contract that reaches all the way into storage. With a hot table behind it, AI output gets instant validation and lands with no DDL.
Part 2: Killing N+1
How one SQL trick cut our latency by 40x
We cut database round-trips from 101 to 1, and latency from 1000ms to 25ms, a 97% improvement. The trick is PostgreSQL's CTE + JSON_AGG.
Part 3: Zero Dirty Reads Lakehouse
Building a trustworthy lakehouse with DuckDB
PostgreSQL handles the present, DuckDB and Parquet handle the past. Anti-Join and a Dirty Set are what keep federated queries clean.
Forma FAQ
Common objections, answered
Online DDL, JSONB, MongoDB, "EAV is an anti-pattern": the questions we hear most, and how Forma addresses them.
中文系列
三篇工程博客，讲透一个为 AI 时代设计的灵活数据存储引擎。
系列介绍：从 EAV 到零脏读的 Lakehouse
Forma 是什么，它解决什么问题
从这里开始了解 Forma 的架构和它要解决的三个核心问题。
第一篇：为什么 EAV 是 AI 时代最被低估的数据模型
JSON Schema + 热表 = AI-Ready 基础设施
配合热表，JSON Schema 可以把类型契约从校验一路带进存储：AI 输出、即时校验、零 DDL 入库。
第二篇：杀死 N+1
一次 SQL 优化如何让延迟从 1 秒降到 25 毫秒
我们把数据库查询次数从 101 次减到 1 次，延迟从 1000ms 降到 25ms。秘诀是 PostgreSQL 的 CTE + JSON_AGG。
第三篇：零脏读的 Serverless 湖仓
我们如何用 DuckDB 解决一致性难题
PostgreSQL 负责当下，DuckDB + Parquet 负责历史。Anti-Join 加 Dirty Set 机制确保联邦查询零脏读。
Forma 中文 FAQ
常见疑问解答
Online DDL、JSONB、MongoDB、"EAV 是反模式"，这些是我们最常被问到的问题，这里给出 Forma 的回应。
番外：优化 EAV 模式的查询性能
完整技术细节
单查询 + JSON 聚合、联邦查询引擎、Serverless 湖仓分层设计，把 EAV 查询性能提升一到两个数量级。
LTSeq: A Fast, Memory-Efficient Engine for Ordered-Sequence Operations
A five-part series on an engine that makes ordered sequences first-class: a Rust kernel built on DataFusion and Arrow, with native support for window functions, sequential grouping, and streaming pattern matching.
- When Data Has Order: A Complete Architecture Guide to the LTSeq Ordered Sequence Algebra Engine: why order matters, and how the engine is put together.
- From Lambda to Bytecode: Inside the LTSeq Expression System: turning Python lambdas into cross-language executable expression trees.
- The 2,400-Line Bytecode Engine: Inside LTSeq's Core Algorithms: streaming linear scan, direct Parquet parallel scanning, and vectorized pattern matching.
- 8.5 MB vs. 571 MB: LTSeq's Performance Engineering Playbook: 1.4x to 1.5x DuckDB throughput at 67x to 112x less memory, validated on ClickBench.
- From Zero to Production: A Practical Guide to LTSeq: runnable examples for sessions, funnels, and multi-table analytics, plus the pitfalls.
Enterprise Ontology
A three-part series on the semantic layer enterprise AI agents are missing.
- The Semantic Blindspot: Why Bigger Models Won't Save Your AI Agent Without Ontology: the failure mode killing enterprise AI pilots, and why no one is benchmarking it.
- Data, Logic, Action: The Ontology Stack Every Enterprise Agent Needs: the architecture that turns data tables into business objects agents can act on.
- How to Build an Enterprise Ontology Without a 2-Year Knowledge Engineering Project: the pragmatic build playbook that ships in weeks.
Harness Engineering
- Your AI Isn't "Stupid," It Just Needs a Better Harness: agents don't fail because models are weak; they fail because systems are undefined.
- Harness Engineering: From Theory to Production: nine questions on building AI agents that don't collapse under real-world conditions.
Strategy & Frameworks
- The Human-AI Partnership Readiness Framework: an operational tool for scoring organizational readiness and setting your AI adoption pace.