PipeQL

Pipelined · Injection-Safe · Polyglot Query Language

📚 Docs & Live Playground · 💿 Install · 🧠 Syntax Reference · 🔌 SDK Usage · ⬇️ Download Binaries

PipeQL is a compiled query language that transpiles to parameterized SQL. You write clean, left-to-right pipelines; the compiler extracts every value into bind parameters at the AST level — making SQL injection mathematically impossible. One query, four databases: PostgreSQL, SQLite, DuckDB, and MySQL.

from orders
| join customers on orders.customer_id == customers.id
| filter orders.status == 'active' and orders.total >= $min
| group [region] (total = sum(orders.total), cnt = count(*))
| filter total > $threshold
| select [region, total, cnt]
| sort [total desc]
| take 10

→ PostgreSQL output:

SELECT region, SUM(orders.total) AS total, COUNT(*) AS cnt
FROM orders
INNER JOIN customers ON (orders.customer_id = customers.id)
WHERE ((orders.status = $1) AND (orders.total >= $2))
GROUP BY region HAVING (SUM(orders.total) > $3)
ORDER BY total DESC LIMIT 10;
-- Parameters: [$1='active', $2=min, $3=threshold]

Every string literal and $param is extracted into a positional bind array at the AST level. No string concatenation. No escaping. No injection.

Why PipeQL?

Problem	PipeQL Solution
🛡️ SQL injection	100% AST-level parameter isolation — impossible to inject
🔀 Dialect lock-in	Write once, compile to Postgres / SQLite / DuckDB / MySQL
↔️ Right-to-left SQL	Clean left-to-right pipeline: `from → filter → select → sort`
🐌 Slow template engines	Native Rust compiler, ~19µs per query
🌍 Language silos	One compiler, 5 SDKs: Rust, JS/TS, Python, C/C++, Go
🧩 Fragmented tooling	Built-in LSP, VS Code extension, tree-sitter grammar, CLI

💿 Install

Rust — CLI & Library

cargo install pipeql-cli          # CLI tool

# Cargo.toml
[dependencies]
pipeql-core = "1.1.6"

JavaScript / TypeScript

npm install @flaxmbot/pipeql

Python

pip install pipeql

Go

⚠️ Prerequisites: the Go binding uses CGO and needs the libpipeql_cffi shared library on your system. It is a library, not a command — use go get (inside a Go module), not go install.

# 1. Build the shared library
git clone https://github.com/Flaxmbot/PipeQL.git && cd PipeQL
cargo build --release -p pipeql-cffi

# 2. Install the shared library
# Linux:
sudo cp target/release/libpipeql_cffi.so /usr/local/lib/ && sudo ldconfig
# macOS:
sudo cp target/release/libpipeql_cffi.dylib /usr/local/lib/
# Windows: copy target/release/pipeql_cffi.dll next to your binary (or add to PATH)

# 3. Add the module — from INSIDE your Go project (any dir with a go.mod):
go get github.com/Flaxmbot/PipeQL/go@latest

#    Or pin a specific release:
#    go get github.com/Flaxmbot/PipeQL/go@v1.1.6

If you see go.mod file not found you are outside a Go module — run go mod init <yourmodule> first, or run the go get from a project that already has a go.mod.

C / C++

Build the shared library, then link against the header:

cargo build --release -p pipeql-cffi
# Header:  crates/pipeql-cffi/include/libpipeql.h
# Library: target/release/libpipeql_cffi.{so,dylib,dll}

gcc demo.c -I./crates/pipeql-cffi/include -L./target/release -lpipeql_cffi -o demo

C package managers: for distributing the C SDK, vcpkg (Microsoft; cross-platform, CMake-native, 2400+ ports) and Conan (decentralized, custom remotes, great for versioned dependency graphs) are the two leading choices. vcpkg is the easiest first step — create a port in the ports/ dir, and vcpkg install pipeql works on Windows/macOS/Linux out of the box.

Pre-built Binaries

Download from GitHub Releases:

Platform	CLI	Shared Library
Linux x64	`pipeql-linux-x86_64`	`libpipeql_cffi.so`
macOS x64	`pipeql-macos-x86_64`	`libpipeql_cffi.dylib`
Windows x64	`pipeql-windows-x86_64.exe`	`pipeql_cffi.dll`

Each release also ships: @flaxmbot/pipeql npm tarball, pipeql PyPI wheel + sdist, pipeql-core crate file, the C SDK header tarball, changelog, and the AI system prompt.

🧠 Syntax Reference

Keywords

All PipeQL reserved keywords:

Keyword	Purpose	Example
`from`	Source table for reads, updates, deletes	`from users`
`into`	Target table for inserts and upserts	`into users`
`as`	Alias a table or column	`from users as u`, `select [name as n]`
`filter`	Row filtering (WHERE / HAVING)	`filter age >= 18`
`select`	Choose output columns	`select [id, name]`
`derive`	Add computed columns	`derive [total = price * qty]`
`join`	Inner join	`join orders on users.id == orders.uid`
`left`	Left outer join modifier	`left join orders on ...`
`right`	Right outer join modifier	`right join roles on ...`
`full`	Full outer join modifier	`full join archive on ...`
`inner`	Explicit inner join modifier	`inner join orders on ...`
`on`	Join condition	`join t on a.id == t.id`
`group`	Group by with aggregates	`group [region] (total = sum(amt))`
`sort`	Order results	`sort [created_at desc]`
`take`	Limit rows	`take 25`
`skip`	Offset rows	`skip 50`
`insert`	Insert values	`insert [name = $name]`
`update`	Update values (requires filter)	`update [name = $name]`
`update all`	Update every row (explicit opt-in)	`update all [plan = 'free']`
`delete`	Delete rows (requires filter)	`delete`
`delete all`	Delete every row (explicit opt-in)	`delete all`
`upsert`	Insert-or-update values	`upsert [id = $id, name = $n]`
`conflict`	Conflict target columns for upsert	`conflict [id]`
`do`	Conflict action for upsert	`do update [name = $n]`
`union`	Combine result sets (distinct)	`... \| union ...`
`all`	Include duplicates in union	`... \| union all ...`
`table`	Create a table (DDL)	`table users [...]`
`and`	Logical AND	`filter a == 1 and b == 2`
`or`	Logical OR	`filter a == 1 or b == 2`
`not`	Logical NOT / negation	`filter not active`
`in`	Set membership test	`filter id in (1, 2, 3)`
`is`	Null check	`filter name is null`
`null`	Null literal	`filter name is not null`
`true`	Boolean true	`filter active == true`
`false`	Boolean false	`filter active == false`
`asc`	Sort ascending	`sort [name asc]`
`desc`	Sort descending	`sort [created_at desc]`

Pipeline Stages

Every PipeQL query starts with a source table and chains stages with |:

from <table> [as <alias>]
| <stage>
| <stage>
| ...

Stage	Syntax	SQL Equivalent
from	`from users`	`FROM users`
from (alias)	`from users as u`	`FROM users u`
filter	`filter age >= 18 and active == true`	`WHERE age >= 18 AND active = TRUE`
select	`select [id, name, email]`	`SELECT id, name, email`
select (alias)	`select [full_name as name]`	`SELECT full_name AS name`
select (star)	`select [*]`	`SELECT *`
derive	`derive [total = price * qty]`	`SELECT , (price qty) AS total`
join	`join orders on users.id == orders.user_id`	`INNER JOIN orders ON ...`
left join	`left join orders on users.id == orders.uid`	`LEFT JOIN orders ON ...`
right join	`right join roles on users.role_id == roles.id`	`RIGHT JOIN roles ON ...`
full join	`full join archive on a.id == archive.id`	`FULL JOIN archive ON ...`
group	`group [region] (total = sum(amount))`	`GROUP BY region`
sort	`sort [created_at desc, name asc]`	`ORDER BY created_at DESC, name ASC`
take	`take 25`	`LIMIT 25`
skip	`skip 50`	`OFFSET 50`

Parameters

Parameters are auto-extracted from the query and converted to dialect-specific placeholders:

Syntax	Description	Postgres	SQLite / DuckDB / MySQL
`$name`	Named parameter	`$1`	`?`
`${name}`	Braced parameter	`$1`	`?`
`'literal'`	String literal (auto-extracted)	`$1`	`?`

from users | filter email == $email and role == 'admin'
-- Postgres: WHERE (email = $1) AND (role = $2)  → params: ["email", "admin"]
-- SQLite:   WHERE (email = ?) AND (role = ?)    → params: ["email", "admin"]

Expressions & Operators

Category	Operators	Example
Comparison	`==` `!=` `<` `<=` `>` `>=`	`filter price >= 10`
Logical	`and` `or` `not`	`filter a == 1 and not b`
Null checks	`is null` `is not null`	`filter name is not null`
Set membership	`in (...)` `not in (...)`	`filter id in (1, 2, 3)`
Subquery	`in (from ... \| select ...)`	`filter id in (from t \| select [id])`
Arithmetic	`+` `-` `*` `/`	`derive [total = price * qty]`
Functions	`count(*)` `sum()` `avg()` `min()` `max()` `coalesce()`	`group [r] (n = count(*))`
Column ref	`table.column`	`filter users.id == orders.uid`
Literals	integers, floats, strings, booleans, null	`42` `3.14` `'text'` `true` `null`

Mutations (DML)

Insert

into users | insert [name = $name, email = $email]

→ INSERT INTO users (name, email) VALUES ($1, $2) RETURNING *;

Update

from users | filter id == $id | update [name = $name, email = $email]

→ UPDATE users SET name = $1, email = $2 WHERE (id = $3);

⚠️ update requires a preceding filter stage — PipeQL enforces this to prevent accidental mass updates.

🔓 Escape hatch: to deliberately update every row, write update all [...] explicitly:
from users | update all [plan = 'free']
→ UPDATE users SET plan = $1; (no WHERE)

If a filter is present alongside all, the WHERE clause still applies.

Delete

from users | filter id == $id | delete

→ DELETE FROM users WHERE (id = $1);

⚠️ delete requires a preceding filter stage — same safety enforcement as update.

🔓 Escape hatch: to deliberately clear a table, write delete all explicitly:
from users | delete all
→ DELETE FROM users; (no WHERE)

If a filter is present alongside all, the WHERE clause still applies.

Upsert (Insert or Update on Conflict)

into users
| upsert [id = $id, name = $name, email = $email]
| conflict [id]
| do update [name = $name, email = $email]

Dialect	Output
Postgres / SQLite / DuckDB	`INSERT INTO users (...) VALUES (...) ON CONFLICT (id) DO UPDATE SET name = $4, email = $5;`
MySQL	`INSERT INTO users (...) VALUES (...) ON DUPLICATE KEY UPDATE name = VALUES(name), email = VALUES(email);`

Union

from active_users | select [id, name]
| union
from archived_users | select [id, name]

→ SELECT id, name FROM active_users UNION SELECT id, name FROM archived_users;

Use union all to include duplicates.

Subqueries

from orders
| filter customer_id in (from vip_customers | select [id])
| select [order_id, total]

→ SELECT order_id, total FROM orders WHERE customer_id IN (SELECT id FROM vip_customers);

DDL (Table Schema)

table users [
  id int primary auto,
  name string not null,
  email string not null unique,
  active bool default true,
  created_at timestamp default '2024-01-01'
]

Type	Aliases	Column Modifiers
`int`	`integer`	`primary`, `auto`, `unique`, `not null`, `default <value>`
`float`	`real`	`primary`, `auto`, `unique`, `not null`, `default <value>`
`string`	`text`	`primary`, `auto`, `unique`, `not null`, `default <value>`
`bool`	`boolean`	`primary`, `auto`, `unique`, `not null`, `default <value>`
`timestamp`	`datetime`	`primary`, `auto`, `unique`, `not null`, `default <value>`

Type Mapping Across Dialects

PipeQL Type	PostgreSQL	SQLite	DuckDB	MySQL
`int` / `integer`	`INTEGER`	`INTEGER`	`INTEGER`	`INT`
`float` / `real`	`DOUBLE PRECISION`	`REAL`	`DOUBLE`	`DOUBLE`
`string` / `text`	`TEXT`	`TEXT`	`VARCHAR`	`VARCHAR(255)`
`bool` / `boolean`	`BOOLEAN`	`INTEGER`	`BOOLEAN`	`BOOLEAN`
`timestamp` / `datetime`	`TIMESTAMP`	`DATETIME`	`TIMESTAMP`	`TIMESTAMP`

Comments

-- This is a line comment
from users | select [id, name]  -- inline comment

Comments are preserved in the lossless AST for IDE tooling.

🔌 SDK Usage

Rust

use pipeql_core::api;

let result = api::compile("from users | filter id == $id | select [name]", "postgres").unwrap();
println!("{}", result.sql);      // SELECT name FROM users WHERE (id = $1);
println!("{:?}", result.params); // ["id"]

JavaScript / TypeScript

import { compile } from '@flaxmbot/pipeql';

const { sql, params } = compile(
  "from notes | filter category == $cat | sort [updated_at desc]",
  "sqlite"
);
console.log(sql);    // SELECT * FROM notes WHERE (category = ?) ORDER BY updated_at DESC;
console.log(params); // ["cat"]

Driver adapters — zero-boilerplate DB wrappers

import { createPipeqlDriver } from '@flaxmbot/pipeql/driver';
import sqlite3 from 'sqlite3';

const db = createPipeqlDriver(new sqlite3.Database('app.db'), { dialect: 'sqlite' });

const rows = await db.query('from users | filter role == $role', { role: 'admin' });
const { lastId, changes } = await db.execute('into notes | insert [title = $title]', { title: 'Hi' });
const newNote = await db.insertAndFetch('into notes | insert $data', req.body);

Python

import pipeql_python as pipeql

res = pipeql.compile("into users | insert [name = $name, email = $email]", "postgres")
print(res["sql"])    # INSERT INTO users (name, email) VALUES ($1, $2) RETURNING *;
print(res["params"]) # ["name", "email"]

from pipeql_python.driver import create_pipeql_driver
import sqlite3

db = create_pipeql_driver(sqlite3.connect('app.db'))
rows = db.query("from users | filter role == $role", {"role": "admin"})

Go

package main

import (
    "fmt"
    "log"
    pipeql "github.com/Flaxmbot/PipeQL/go"
)

func main() {
    res, err := pipeql.Compile("from users | filter age >= $min | select [id, name]", "postgres")
    if err != nil { log.Fatal(err) }
    fmt.Println("SQL:", res.SQL)       // SELECT id, name FROM users WHERE (age >= $1);
    fmt.Println("Params:", res.Params) // ["min"]
}

C

#include <stdio.h>
#include "libpipeql.h"

int main() {
    PipeqlError err = {0};
    PipeqlResult* res = pipeql_compile("from users | filter id == $id", "postgres", &err);
    if (!res) { fprintf(stderr, "Error: %s\n", err.message); return 1; }
    printf("SQL: %s\n", res->sql);
    pipeql_result_free(res);
    return 0;
}

CLI

pipeql compile "from users | take 10" --dialect postgres
pipeql compile "from users | filter id == $id" --dialect sqlite
pipeql parse "from users | select [id, name]"
pipeql dialects
pipeql --version

Optional: Fluent builder (programmatic composition)

The string DSL above is PipeQL's primary interface. For composing queries in code — conditional or looped pipeline stages, or object-style inserts — every SDK additionally ships an optional fluent builder. A builder query and a hand-written string query are provably identical: builders assemble the exact same source string and hand it to the same compiler. Object inserts accept key → value objects and auto-generate $b0, $b1, ... bind parameters.

// Rust — pipeql_core::builder
use pipeql_core::builder::{Query, Value};

let q = Query::from("notes")
    .filter("is_archived == 0")
    .sort(["created_at desc"])
    .take(10);
let compiled = q.compile("postgres").unwrap();

let ins = Query::into_("notes").insert([("title", Value::Str("Hi".into()))]);
// source: "into notes | insert [title = $b0]"  values: [("title", "Hi")]

// JavaScript — @flaxmbot/pipeql/builder (also works through a driver: db.query(q))
import { PipeQL } from '@flaxmbot/pipeql/builder';

const q = PipeQL.from('notes')
  .filter('is_archived == 0')
  .sort(['created_at desc'])
  .take(10);
const { sql, params } = await q.compile('sqlite');

const ins = PipeQL.into('notes').insert({ title: 'Hi', flag: 1 });
// source: "into notes | insert [title = $b0, flag = $b1]"  values: { b0: 'Hi', b1: 1 }

# Python — pipeql_python.builder (also works through a driver: db.query(q))
from pipeql_python.builder import PipeQL

q = (PipeQL.from_("notes")
     .filter("is_archived == 0")
     .sort(["created_at desc"])
     .take(10))
result = q.compile("postgres")
rows = db.query(q)

ins = PipeQL.into_("notes").insert({"title": "Hi", "flag": 1})
# source: "into notes | insert [title = $b0, flag = $b1]"  values: {"b0": "Hi", "b1": 1}

// Go — pipeql (maps are sorted for deterministic SQL; PairsOf keeps exact order)
q := pipeql.From("notes").
    Filter("is_archived == 0").
    Sort([]string{"created_at desc"}).
    Take(10)
res, err := q.Compile("postgres")

ins := pipeql.Into("notes").Insert(pipeql.PairsOf("title", "Hi", "flag", 1))

// C — libpipeql (every stage returns the handle for chaining)
PipeqlQuery* q = pipeql_query_from("notes");
q = pipeql_query_filter(q, "is_archived == 0");
q = pipeql_query_sort(q, "created_at desc");
q = pipeql_query_take(q, 10);
PipeqlResult* built = pipeql_query_compile(q, "postgres", &err);
printf("SQL: %s\n", built->sql);
pipeql_result_free(built);
pipeql_query_free(q);

When to use the builder: conditional or looped pipeline stages, object-style inserts, and passing a query object straight into a driver. For one-shot queries, the string DSL is shorter and equally safe.

Ecosystem

Component	Description
`pipeql-core`	Core compiler: lexer → parser → AST → codegen
`pipeql-cli`	Command-line tool
`pipeql-wasm`	WebAssembly target for browsers
`pipeql-python`	Python binding (PyO3, ABI3) → `pipeql` on PyPI
`pipeql-cffi`	C ABI shared library
`pipeql-lsp`	Language Server Protocol
`js/`	JavaScript/TypeScript SDK (`@flaxmbot/pipeql`)
`go/`	Go binding (CGO) — `github.com/Flaxmbot/PipeQL/go`
`python/`	Python package + driver adapters
`docs-web/`	Interactive documentation + WASM playground
`extensions/`	VS Code extension
`tree-sitter-pipeql/`	Tree-sitter grammar

Compiler Architecture

Source Text ──→ Lexer ──→ Tokens ──→ Parser ──→ AST ──→ Codegen ──→ SQL + Params
                          │                     │                    │
                          │                     │                    ├─ PostgreSQL ($1, $2)
                          │                     │                    ├─ SQLite     (?, ?)
                          │                     │                    ├─ DuckDB     (?, ?)
                          │                     │                    └─ MySQL      (?, ?)
                          │                     │
                          │                     └─ Lossless AST (spans + comments)
                          │
                          └─ Character-level span tracking

Lexer — hand-written tokenizer with exact character positions for IDE support
Parser — Pratt parser producing a lossless abstract syntax tree
Codegen — walks the AST, extracts all values into bind parameters, emits dialect-specific SQL

All language bindings are thin wrappers calling the Rust core through pipeql-core's API.

Safety: #![deny(unsafe_code)] enforced across the entire compiler core.

Error Messages

PipeQL provides compiler-grade error messages with exact positions and actionable suggestions:

Error at line 1, col 1: Unknown keyword 'selct'
  hint: Did you mean 'select'?

Error at line 1, col 35: 'update' requires a preceding 'filter' stage
  hint: Add a filter to prevent accidental mass updates: from <table> | filter ... | update [...] (or write `update all [...]` to explicitly opt in)

Error at line 1, col 15: Unclosed string literal
  hint: Add a closing single quote (') to terminate the string.

Features: Levenshtein-based fuzzy keyword matching, contextual hints, unclosed string/subquery detection, duplicate column detection, empty pipeline errors, filter-before-mutate enforcement.

AI & LLM Integration

PipeQL ships with an optimized System Prompt for code generation with LLMs (GPT-4, Claude, Gemini, etc.):

# Python
import pipeql_python
system_prompt = pipeql_python.SYSTEM_PROMPT

// JavaScript — bundled in npm package
import prompt from '@flaxmbot/pipeql/ai/system_prompt.md';

Direct download: ai/system_prompt.md

Contributing

git clone https://github.com/Flaxmbot/PipeQL.git && cd PipeQL
cargo test --workspace              # Run all tests
cargo bench -p pipeql-core          # Benchmarks
cargo build --release -p pipeql-cli # Build CLI

License

MIT © Flaxmbot

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
ai		ai
crates		crates
docs-web		docs-web
docs		docs
examples		examples
extensions/vscode-pipeql		extensions/vscode-pipeql
go		go
js		js
python		python
tree-sitter-pipeql		tree-sitter-pipeql
vcpkg		vcpkg
.gitignore		.gitignore
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
LICENSE		LICENSE
README.md		README.md
logo.png		logo.png
logo.svg		logo.svg
system_prompt.md		system_prompt.md

Latest commit

History

Folders and files

Repository files navigation

PipeQL

Pipelined · Injection-Safe · Polyglot Query Language

Why PipeQL?

💿 Install

Rust — CLI & Library

JavaScript / TypeScript

Python

Go

C / C++

Pre-built Binaries

🧠 Syntax Reference

Keywords

Pipeline Stages

Parameters

Expressions & Operators

Mutations (DML)

Insert

Update

Delete

Upsert (Insert or Update on Conflict)

Union

Subqueries

DDL (Table Schema)

Type Mapping Across Dialects

Comments

🔌 SDK Usage

Rust

JavaScript / TypeScript

Driver adapters — zero-boilerplate DB wrappers

Python

Go

C

CLI

Optional: Fluent builder (programmatic composition)

Ecosystem

Compiler Architecture

Error Messages

AI & LLM Integration

Contributing

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages