Forma Engineering Blog

Choose your language: 中文 | English

English Series

A three-part engineering series on building flexible, high-performance data storage for AI applications.

Series Introduction: From EAV to Zero-Dirty-Read Lakehouse

What Forma is and what problems it solves

Start here for an overview of the architecture and the three core problems behind it.

Part 1: Why EAV is the Most Underrated Data Model for AI

JSON Schema + hot table = AI-ready infrastructure

JSON Schema is the type contract that reaches all the way into storage. With a hot table behind it, AI output gets instant validation and lands with no DDL.

Part 2: Killing N+1

How one SQL trick cut our latency by 40x

We cut database round-trips from 101 to 1, and latency from 1000ms to 25ms, a 97% improvement. The trick is PostgreSQL's CTE + JSON_AGG.

Part 3: Zero Dirty Reads Lakehouse

Building a trustworthy lakehouse with DuckDB

PostgreSQL handles the present, DuckDB and Parquet handle the past. Anti-Join and a Dirty Set are what keep federated queries clean.

Forma FAQ

Common objections, answered

Online DDL, JSONB, MongoDB, "EAV is an anti-pattern": the questions we hear most, and how Forma addresses them.

中文系列

三篇工程博客，讲透一个为 AI 时代设计的灵活数据存储引擎。

系列介绍：从 EAV 到零脏读的 Lakehouse

Forma 是什么，它解决什么问题

从这里开始了解 Forma 的架构和它要解决的三个核心问题。

第一篇：为什么 EAV 是 AI 时代最被低估的数据模型

JSON Schema + 热表 = AI-Ready 基础设施

配合热表，JSON Schema 可以把类型契约从校验一路带进存储：AI 输出、即时校验、零 DDL 入库。

第二篇：杀死 N+1

一次 SQL 优化如何让延迟从 1 秒降到 25 毫秒

我们把数据库查询次数从 101 次减到 1 次，延迟从 1000ms 降到 25ms。秘诀是 PostgreSQL 的 CTE + JSON_AGG。

第三篇：零脏读的 Serverless 湖仓

我们如何用 DuckDB 解决一致性难题

PostgreSQL 负责当下，DuckDB + Parquet 负责历史。Anti-Join 加 Dirty Set 机制确保联邦查询零脏读。

Forma 中文 FAQ

常见疑问解答

Online DDL、JSONB、MongoDB、"EAV 是反模式"，这些是我们最常被问到的问题，这里给出 Forma 的回应。

番外：优化 EAV 模式的查询性能

完整技术细节

单查询 + JSON 聚合、联邦查询引擎、Serverless 湖仓分层设计，把 EAV 查询性能提升一到两个数量级。

Why Forma?

Traditional databases weren't built for the AI era. When your AI Agent outputs 12 fields today and 30 fields tomorrow, waiting 3-7 days for DDL approval isn't an option.

Forma's answer is a modern take on the EAV pattern:

Problem	Traditional DB	Forma
New field	ALTER TABLE (days)	JSON Schema update (seconds)
Schema change	Downtime required	Zero downtime
AI output	Manual adaptation	Direct JSON Schema mapping
N+1 queries	101 round-trips	1 round-trip
Historical data	Same table, same cost	Cold storage on S3

Quick Start

# Clone the repository
git clone https://github.com/Lychee-Technology/forma.git
cd forma

# Start the development environment
make dev

# Run tests
make test

Quick Reference

Problem	Solution	Key Metric
Schema flexibility	EAV + JSON Schema + Hot Table	Zero DDL, instant field changes
N+1 queries	CTE + JSON_AGG	101→1 queries, 97% latency reduction
Historical data scale	DuckDB + CDC + Parquet	Zero dirty reads, Serverless cost