chore: import upstream snapshot with attribution
Component Security Validation / Security Audit (push) Has been cancelled
Deploy to Cloudflare Pages / deploy (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 12:38:58 +08:00
commit bb5c75ce05
8824 changed files with 1946442 additions and 0 deletions
@@ -0,0 +1,67 @@
---
name: alloydb-basics
description: Manages clusters, instances, and backups for AlloyDB for PostgreSQL, and integrates with AlloyDB MCP tools for automated database operations including AI-powered search and vector capabilities.
source: google/skills (Apache 2.0)
---
# AlloyDB Basics
AlloyDB for PostgreSQL is a managed, PostgreSQL-compatible database service
designed for enterprise-grade performance and availability. It utilizes a
disaggregated compute and storage architecture to scale resources independently.
It also provides AlloyDB AI, a collection of features that includes AI-powered
search (vector, hybrid search, and AI functions), natural language capabilities,
conversational analytics, and inference features like forecasting and model
endpoint management to help developers build AI apps faster.
## Quick Start
1. **Enable the AlloyDB API:**
```bash
gcloud services enable alloydb.googleapis.com --quiet
```
2. **Create a Cluster:**
```bash
gcloud alloydb clusters create my-cluster --region=us-central1 \
--password=my-password --network=my-vpc \
--quiet
```
*Note: For production, we recommend using IAM database authentication
instead of passwords. If passwords must be used, use secure secret
management (e.g., Secret Manager) instead of passing passwords in
cleartext.*
3. **Create a Primary Instance:**
```bash
gcloud alloydb instances create my-primary --cluster=my-cluster \
--region=us-central1 --instance-type=PRIMARY --cpu-count=2 \
--quiet
```
## Reference Directory
- [Core Concepts](references/core-concepts.md): Architecture, disaggregated
storage, and performance features.
- [CLI Usage](references/cli-usage.md): Essential `gcloud alloydb` commands
for cluster and instance management.
- [Client Libraries & Connectors](references/client-library-usage.md):
Connecting to AlloyDB using Python, Java, Node.js, and Go.
- [MCP Usage](references/mcp-usage.md): Using the AlloyDB remote MCP server
and Gemini CLI extension.
- [Infrastructure as Code](references/iac-usage.md): Terraform
configuration and deployment examples.
- [IAM & Security](references/iam-security.md): Predefined roles, service
agents, and database authentication.
*If you need product information not found in these references, use the
Developer Knowledge MCP server `search_documents` tool.*
@@ -0,0 +1,37 @@
# AlloyDB CLI Usage
AlloyDB resources are managed using the `gcloud alloydb` command group.
## Clusters
1. Create a cluster: `gcloud alloydb clusters create CLUSTER_ID --region=REGION
--password=PASSWORD`
2. List clusters: `gcloud alloydb clusters list --region=REGION`
3. Get cluster info: `gcloud alloydb clusters describe CLUSTER_ID
--region=REGION`
4. Delete a cluster: `gcloud alloydb clusters delete CLUSTER_ID --region=REGION`
## Instances
1. Create a primary instance: `gcloud alloydb instances create INSTANCE_ID
--cluster=CLUSTER_ID --region=REGION --instance-type=PRIMARY --cpu-count=8`
2. Create a read pool instance: `gcloud alloydb instances create INSTANCE_ID
--cluster=CLUSTER_ID --region=REGION --instance-type=READ_POOL
--read-pool-node-count=2 --cpu-count=2`
3. List instances: `gcloud alloydb instances list --cluster=CLUSTER_ID
--region=REGION`
4. Restart an instance: `gcloud alloydb instances restart INSTANCE_ID
--cluster=CLUSTER_ID --region=REGION`
## Backups
1. Create a backup: `gcloud alloydb backups create BACKUP_ID
--cluster=CLUSTER_ID --region=REGION`
2. List backups: `gcloud alloydb backups list --region=REGION`
@@ -0,0 +1,187 @@
# AlloyDB Client Libraries & Connectors
Google Cloud provides various ways to connect to AlloyDB idiomatically from
different programming languages. We optionally provide Client Libraries and
Connectors to facilitate secure authentication and connection from your clients
to your AlloyDB instances. These tools handle the management of SSL
certificates, firewall rules, and IAM Auth token automation.
## AlloyDB Language Connectors
Language connectors are libraries for Python, Java, and Go designed for
developers who prefer an integrated, driver-level experience over the
operational overhead of managing the Auth Proxy as a separate binary.
### Python
- **Installation:**
```bash
pip install "google-cloud-alloydb-connector[pg8000]" sqlalchemy
```
- **Usage Example:**
```python
import sqlalchemy
from google.cloud.alloydbconnector import Connector
INSTANCE_URI = "projects/MY_PROJECT/locations/MY_REGION/clusters/MY_CLUSTER/instances/MY_INSTANCE"
with Connector() as connector:
pool = sqlalchemy.create_engine(
"postgresql+pg8000://",
creator=lambda: connector.connect(
INSTANCE_URI,
"pg8000",
user="my-user",
password="my-password",
db="my-db",
),
)
with pool.connect() as conn:
result = conn.execute(sqlalchemy.text("SELECT NOW()")).fetchone()
print(result)
```
### Java
- **Maven Dependency:**
```xml
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>alloydb-jdbc-connector</artifactId>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
</dependency>
<dependency>
<groupId>com.zaxxer</groupId>
<artifactId>HikariCP</artifactId>
</dependency>
```
- **Configuring a Connection Pool:**
We recommend using HikariCP for connection pooling. To use HikariCP with the
Java Connector, you will need to set the usual properties (e.g., JDBC URL,
username, password, etc) and you will need to set two Connector specific
properties:
* `socketFactory` should be set to
`com.google.cloud.alloydb.SocketFactory`
* `alloydbInstanceName` should be set to the AlloyDB instance you want to
connect to, e.g.:
`projects/<PROJECT>/locations/<REGION>/clusters/<CLUSTER>/instances/<INSTANCE>`
Basic configuration of a data source looks like this:
```java
import com.zaxxer.hikari.HikariConfig;
import com.zaxxer.hikari.HikariDataSource;
public class ExampleApplication {
private HikariDataSource dataSource;
public HikariDataSource getDataSource() {
HikariConfig config = new HikariConfig();
// There is no need to set a host on the JDBC URL
// since the Connector will resolve the correct IP address.
config.setJdbcUrl(String.format("jdbc:postgresql:///%s", System.getenv("ALLOYDB_DB")));
config.setUsername(System.getenv("ALLOYDB_USER"));
config.setPassword(System.getenv("ALLOYDB_PASS"));
// Tell the driver to use the AlloyDB Java Connector's SocketFactory
// when connecting to an instance/
config.addDataSourceProperty("socketFactory",
"com.google.cloud.alloydb.SocketFactory");
// Tell the Java Connector which instance to connect to.
config.addDataSourceProperty("alloydbInstanceName",
System.getenv("ALLOYDB_INSTANCE_NAME"));
dataSource = new HikariDataSource(config);
return dataSource;
}
// Use DataSource as usual ...
}
```
See [end to end
test](https://github.com/GoogleCloudPlatform/alloydb-java-connector/blob/main/jdbc/postgres/src/test/java/com/google/cloud/alloydb/postgres/PgJdbcIntegrationTests.java)
for a full example.
See [About Pool
Sizing](https://github.com/brettwooldridge/HikariCP/wiki/About-Pool-Sizing)
for useful guidance on getting the best performance from a connection pool.
### Go
- **Installation:**
```bash
go get cloud.google.com/go/alloydbconn
```
- **Usage Example:**
```go
package main
import (
"database/sql"
"fmt"
"log"
"cloud.google.com/go/alloydbconn/driver/pgxv5"
)
func main() {
// Register the AlloyDB driver with the name "alloydb"
// Uses Private IP by default. See Network Options below for details.
cleanup, err := pgxv5.RegisterDriver("alloydb")
if err != nil {
log.Fatal(err)
}
defer cleanup()
// Instance URI format:
// projects/PROJECT/locations/REGION/clusters/CLUSTER/instances/INSTANCE
db, err := sql.Open("alloydb", fmt.Sprintf(
"host=%s user=%s password=%s dbname=%s sslmode=disable",
"projects/my-project/locations/us-central1/clusters/my-cluster/instances/my-instance",
"my-user",
"my-password",
"my-db",
))
if err != nil {
log.Fatal(err)
}
defer db.Close()
var greeting string
if err := db.QueryRow("SELECT 'Hello, AlloyDB!'").Scan(&greeting); err != nil {
log.Fatal(err)
}
fmt.Println(greeting)
}
```
## Standard PostgreSQL Drivers
Since AlloyDB is PostgreSQL-compatible, you can also use standard drivers:
- **Python:** `psycopg2`, `asyncpg`, `pg8000`
- **Java:** `PostgreSQL JDBC Driver`
- **Go:** `lib/pq`, `jackc/pgx`
For more details, see: [AlloyDB
Connectors](https://cloud.google.com/alloydb/docs/connect-external).
@@ -0,0 +1,50 @@
# AlloyDB Core Concepts
AlloyDB for PostgreSQL is a fully managed, PostgreSQL-compatible database
service designed for high performance, scale, and availability. It is built on
top of a cloud-native storage engine that separates compute from storage,
allowing for efficient scaling and high availability.
AlloyDB is ideal for enterprise-grade transactional workloads, such as ERP or
CRM systems, as well as for analytical workloads that benefit from its columnar
engine, and vector workloads using its [vector search
capabilities](https://docs.cloud.google.com/alloydb/docs/ai/perform-vector-search).
## Regional Availability
AlloyDB is a regional service. A cluster consists of a primary instance and
optional read pool instances, all of which are located in the same region. The
storage is replicated across multiple zones within the region to ensure high
availability.
## AlloyDB Auth Proxy
The [AlloyDB Auth
Proxy](https://cloud.google.com/alloydb/docs/auth-proxy/connect) is a standalone
tool that can be deployed in any environment, and works by opening a local
socket and proxying connections to your AlloyDB instance.
## Connectivity Options
### Private vs Public IP
When connecting to AlloyDB, you can use either a Private IP or a Public IP:
- **Private IP:** Your client must be deployed either in the same VPC network
as your AlloyDB cluster (when using PSA), or have a PSC endpoint in your VPC
(when using PSC) to connect directly using Private IP. For indirect methods
of connecting outside your VPC, see [Enable private services
access](https://cloud.google.com/alloydb/docs/configure-connectivity).
- **Public IP:** If enabled on your instance, you can connect from outside the
VPC network.
## Connection Pooling
For production workloads, use connection poolers like **PgBouncer** (integrated
in AlloyDB) to manage high numbers of concurrent connections efficiently.
## Pricing
For up-to-date pricing information, visit the official [AlloyDB
Pricing](https://cloud.google.com/alloydb/pricing) page. Pricing is based on the
number of vCPUs and memory for each instance, as well as the storage used.
@@ -0,0 +1,135 @@
# AlloyDB Infrastructure as Code Usage
AlloyDB resources can be managed using Terraform via the Google Cloud Provider,
or via Kubernetes Config Connector (KCC).
## Terraform
### Resources
1. `google_alloydb_cluster`: Manages an AlloyDB cluster.
2. `google_alloydb_instance`: Manages an AlloyDB instance within a cluster.
### Example
```terraform
data "google_project" "project" {}
resource "google_compute_network" "default" {
name = "alloydb-network"
}
resource "google_compute_global_address" "private_ip_alloc" {
name = "alloydb-cluster"
address_type = "INTERNAL"
purpose = "VPC_PEERING"
prefix_length = 16
network = google_compute_network.default.id
}
resource "google_service_networking_connection" "vpc_connection" {
network = google_compute_network.default.id
service = "servicenetworking.googleapis.com"
reserved_peering_ranges = [google_compute_global_address.private_ip_alloc.name]
}
resource "google_alloydb_cluster" "default" {
cluster_id = "alloydb-cluster"
location = "us-central1"
network_config {
network = google_compute_network.default.id
}
initial_user {
password = "alloydb-cluster"
}
deletion_protection = false
}
resource "google_alloydb_instance" "default" {
cluster = google_alloydb_cluster.default.name
instance_id = "alloydb-instance"
instance_type = "PRIMARY"
machine_config {
cpu_count = 2
}
depends_on = [google_service_networking_connection.vpc_connection]
}
```
For more information, see the [Google Provider
Reference](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/alloydb_cluster).
## Kubernetes Config Connector (KCC)
### Resources
1. `AlloyDBCluster`: Manages an AlloyDB cluster.
2. `AlloyDBInstance`: Manages an AlloyDB instance within a cluster.
### Example
```yaml
apiVersion: compute.cnrm.cloud.google.com/v1beta1
kind: ComputeNetwork
metadata:
name: alloydb-network-kcc
spec:
routingMode: REGIONAL
autoCreateSubnetworks: false
---
apiVersion: compute.cnrm.cloud.google.com/v1beta1
kind: ComputeAddress
metadata:
name: alloydb-kcc-addr
spec:
location: global
addressType: INTERNAL
purpose: VPC_PEERING
prefixLength: 16
networkRef:
name: alloydb-network-kcc
---
apiVersion: servicenetworking.cnrm.cloud.google.com/v1beta1
kind: ServiceNetworkingConnection
metadata:
name: alloydb-vpc-connection-kcc
spec:
networkRef:
name: alloydb-network-kcc
service: servicenetworking.googleapis.com
reservedPeeringRanges:
- name: alloydb-kcc-addr
---
apiVersion: alloydb.cnrm.cloud.google.com/v1beta1
kind: AlloyDBCluster
metadata:
name: alloydb-cluster-kcc
spec:
location: us-central1
networkConfig:
networkRef:
name: alloydb-network-kcc
initialUser:
password:
valueFrom:
secretKeyRef:
name: alloydb-secret
key: password
---
apiVersion: alloydb.cnrm.cloud.google.com/v1beta1
kind: AlloyDBInstance
metadata:
name: alloydb-instance-kcc
spec:
clusterRef:
name: alloydb-cluster-kcc
instanceType: PRIMARY
machineConfig:
cpuCount: 2
```
For more information, see the [Config Connector resources](https://docs.cloud.google.com/config-connector/docs/reference/overview).
@@ -0,0 +1,93 @@
# AlloyDB IAM & Security
AlloyDB utilizes Google Cloud Identity and Access Management (IAM) to provide
granular access control and robust security features.
## Predefined IAM Roles
The following table describes the predefined roles available for AlloyDB:
| Role Name | Usage |
| :--- | :--- |
| `roles/alloydb.admin` | Full control of all AlloyDB resources. |
| `roles/alloydb.client` | Connectivity access to AlloyDB instances. |
| `roles/alloydb.databaseUser` | Authenticated database-user access to instances. |
| `roles/alloydb.viewer` | Read-only access to all AlloyDB resources. |
## Secure Connectivity
1. **Network Security:**
- **Private IP:** Keeps traffic internal to Google Cloud.
- **Private Service Connect (PSC):** Recommended for new
configurations. Offers enhanced security, better IP management, and
flexible multi-VPC topologies without peering.
- **Private Services Access (PSA):** Uses VPC peering.
- **Public IP:** Allows connections from outside GCP.
- **ALWAYS** use with **Authorized Networks** to restrict access to
specific IP ranges.
- **NEVER** use `0.0.0.0/0` in Authorized Networks.
- **VPC Service Controls (VPC-SC):** Define security perimeters around
your AlloyDB instances to prevent data exfiltration.
1. **Encryption:**
- **In Transit:** TLS encryption is enforced by default for all
connections.
- **At Rest:** Data is always encrypted, using Google-managed keys by
default. Customer-Managed Encryption Keys (CMEK) are supported for
greater control.
1. **Authentication & Authorization:**
- **IAM Authentication:** **Recommended.** Uses Google Cloud Identity and
Access Management (IAM) principals (users/service accounts) and
short-lived OAuth tokens. Centralizes access control and avoids database
passwords.
- **Built-in Database Authentication:** Standard PostgreSQL
username/password. Use strong password policies if this method is
chosen.
- **Database Authorization:** Standard PostgreSQL GRANT / REVOKE
statements manage permissions within the database, regardless of the
authentication method.
1. **Connection Methods:**
- **Direct Connection:** Standard PostgreSQL connection.
- **AlloyDB Connectors (Auth Proxy & Language Libraries):** **Highly
Recommended for security.**
- Provide connection-level **IAM Authorization**.
- Automatically use **mTLS** (Mutual TLS) for encrypted and
authenticated connections.
- Simplify using IAM Database Authentication (tokens are handled
automatically).
- Essential for secure Public IP access without wide open Authorized
Networks.
- Best practice: Run Auth Proxy as a sidecar to the application.
**Key Recommendations for Agents:**
- **Prioritize Private IP:** Guide users to use Private IP (especially PSC)
over Public IP whenever their application is within GCP.
- **Push AlloyDB Connectors:** Strongly recommend AlloyDB Connectors (Auth
Proxy or language libraries) because they enhance security through IAM
connection authorization and mTLS, especially crucial for Public IP.
- **IAM Authentication is Preferred:** Encourages centralized management and
token-based auth.
- **Secure Public IP:** If Public IP is necessary, stress the absolute need
for tightly restricted Authorized Networks.
- **Leverage Cloud Security Tools:** Remind users to use VPC-SC and Security
Command Center for monitoring and policy enforcement.
## Data Security
- **Encryption at Rest:** All data is encrypted by default. Use Customer-Managed
Encryption Keys (CMEK) for greater control.
- **IAM Database Authentication:** Authenticate to the database using IAM
identities (users or service accounts) instead of static passwords.
## Service Agents
AlloyDB uses a managed service agent
(`service-PROJECT_NUMBER@gcp-sa-alloydb.iam.gserviceaccount.com`) to manage
resources like storage and backups. Ensure this agent has the necessary
permissions in your project.
For more information, see: [Security, privacy, risk, and compliance for AlloyDB for PostgreSQL](https://docs.cloud.google.com/alloydb/docs/security-privacy-compliance).
@@ -0,0 +1,29 @@
# AlloyDB MCP Usage
AlloyDB supports a remote Model Context Protocol (MCP) server, allowing AI
applications to interact with AlloyDB resources.
## Endpoint
The AlloyDB MCP server endpoint is regional:
`https://alloydb.REGION.rep.googleapis.com/mcp`
Replace `REGION` with the regional location of the endpoint (e.g.,
`us-central1`).
## Setup and Authentication
1. Enable the AlloyDB API in your project.
2. Grant the `roles/mcp.toolUser` role to the principal making the tool calls.
3. Configure your MCP host to point to the regional endpoint.
For more details, see the [Use the AlloyDB remote MCP
server](https://cloud.google.com/alloydb/docs/ai/use-alloydb-mcp) guide.
## Resources
- [AlloyDB MCP Reference](https://cloud.google.com/alloydb/docs/reference/mcp)
- [MCP Toolbox](https://mcp-toolbox.dev/): An open-source alternative to the remote MCP server that runs on a local machine or IDE.
- [MCP Toolbox AlloyDB Integration](https://mcp-toolbox.dev/integrations/alloydb/source/)
- [Configure your MCP client](https://docs.cloud.google.com/alloydb/docs/connect-ide-using-mcp-toolbox#configure-your-mcp-client)
- For additional specialized skills including health auditing, performance monitoring, and lifecycle management, install the [AlloyDB for PostgreSQL](https://github.com/gemini-cli-extensions/alloydb) Gemini CLI extension or Claude Plugin.
@@ -0,0 +1,98 @@
---
name: bigquery-basics
description: Manages datasets, tables, and jobs in BigQuery, and integrates with BigQuery ML and Gemini for advanced data analytics and AI-driven insights. Use for SQL queries, resource management, data ingestion, or AI applications on BigQuery.
source: google/skills (Apache 2.0)
---
# BigQuery Basics
BigQuery is a serverless, AI-ready data platform that enables high-speed
analysis of large datasets using SQL and Python. Its disaggregated architecture
separates compute and storage, allowing them to scale independently while
providing built-in machine learning, geospatial analysis, and business
intelligence capabilities.
## Setup and Basic Usage
1. **Enable the BigQuery API:**
```bash
gcloud services enable bigquery.googleapis.com --quiet
```
2. **Create a Dataset:**
```bash
bq mk --dataset --location=US my_dataset
```
3. **Create a Table:**
Create a file named `schema.json` with your table schema:
```json
[
{
"name": "name",
"type": "STRING",
"mode": "REQUIRED"
},
{
"name": "post_abbr",
"type": "STRING",
"mode": "NULLABLE"
}
]
```
Then create the table with the `bq` tool:
```bash
bq mk --table my_dataset.mytable schema.json
```
4. **Run a Query:**
```bash
bq query --use_legacy_sql=false \
'SELECT name FROM `bigquery-public-data.usa_names.usa_1910_2013` \
WHERE state = "TX" LIMIT 10'
```
## Reference Directory
- [Core Concepts](references/core-concepts.md): Storage types, analytics
workflows, and BigQuery Studio features.
- [CLI Usage](references/cli-usage.md): Essential `bq` command-line tool
operations for managing data and jobs.
- [Client Libraries](references/client-library-usage.md): Using Google Cloud
client libraries for Python, Java, Node.js, and Go.
- [MCP Usage](references/mcp-usage.md): Using the BigQuery remote MCP server and
Gemini CLI extension.
- [Infrastructure as Code](references/iac-usage.md): Terraform examples for
datasets, tables, and reservations.
- [IAM & Security](references/iam-security.md): Roles, permissions, and data
governance best practices.
*If you need product information not found in these references, use the
Developer Knowledge MCP server `search_documents` tool.*
## Related Skills
- [BigQuery AI & ML Skill](https://github.com/google/adk-python/tree/main/src/google/adk/tools/bigquery/skills/bigquery-ai-ml):
SKILL.md file for BigQuery AI and ML capabilities.
- [BigQuery AI & ML References](https://github.com/google/adk-python/tree/main/src/google/adk/tools/bigquery/skills/bigquery-ai-ml/references):
Reference files published for the BigQuery AI and ML skill.
- [bigquery_ai_classify.md](https://github.com/google/adk-python/blob/main/src/google/adk/tools/bigquery/skills/bigquery-ai-ml/references/bigquery_ai_classify.md)
- [bigquery_ai_detect_anomalies.md](https://github.com/google/adk-python/blob/main/src/google/adk/tools/bigquery/skills/bigquery-ai-ml/references/bigquery_ai_detect_anomalies.md)
- [bigquery_ai_forecast.md](https://github.com/google/adk-python/blob/main/src/google/adk/tools/bigquery/skills/bigquery-ai-ml/references/bigquery_ai_forecast.md)
- [bigquery_ai_generate.md](https://github.com/google/adk-python/blob/main/src/google/adk/tools/bigquery/skills/bigquery-ai-ml/references/bigquery_ai_generate.md)
- [bigquery_ai_generate_bool.md](https://github.com/google/adk-python/blob/main/src/google/adk/tools/bigquery/skills/bigquery-ai-ml/references/bigquery_ai_generate_bool.md)
- [bigquery_ai_generate_double.md](https://github.com/google/adk-python/blob/main/src/google/adk/tools/bigquery/skills/bigquery-ai-ml/references/bigquery_ai_generate_double.md)
- [bigquery_ai_generate_int.md](https://github.com/google/adk-python/blob/main/src/google/adk/tools/bigquery/skills/bigquery-ai-ml/references/bigquery_ai_generate_int.md)
- [bigquery_ai_if.md](https://github.com/google/adk-python/blob/main/src/google/adk/tools/bigquery/skills/bigquery-ai-ml/references/bigquery_ai_if.md)
- [bigquery_ai_score.md](https://github.com/google/adk-python/blob/main/src/google/adk/tools/bigquery/skills/bigquery-ai-ml/references/bigquery_ai_score.md)
- [bigquery_ai_search.md](https://github.com/google/adk-python/blob/main/src/google/adk/tools/bigquery/skills/bigquery-ai-ml/references/bigquery_ai_search.md)
- [bigquery_ai_similarity.md](https://github.com/google/adk-python/blob/main/src/google/adk/tools/bigquery/skills/bigquery-ai-ml/references/bigquery_ai_similarity.md)
@@ -0,0 +1,111 @@
# BigQuery CLI Usage
The `bq` command-line tool is used to interact with BigQuery for managing
resources and running jobs.
## Basic Syntax
```bash
bq COMMAND [FLAGS] [ARGUMENTS]
```
## Essential Commands
### Dataset Management
- **Create a dataset:**
```bash
bq mk --dataset --location=us my_dataset
```
- **List datasets:**
```bash
bq ls --project_id my_project
```
### Table Management
- **Create a table from a schema file:**
```bash
bq mk --table my_dataset.my_table schema.json
```
- **Copy a table within or across datasets:**
```bash
bq cp my_dataset.my_table my_other_dataset.my_table_copy
```
- **Create a table snapshot (read-only copy):**
```bash
bq cp --snapshot --no_clobber my_dataset.my_table my_other_dataset.my_table_snapshot
```
- **Load data from Cloud Storage (CSV):**
```bash
bq load --source_format=CSV my_dataset.my_table gs://my-bucket/data.csv
```
- **Stream data into a table from a newline-delimited JSON file:**
```bash
bq insert my_dataset.my_table data.json
```
- **Delete a table:**
```bash
bq rm -f my_dataset.my_table
```
### Querying Data
- **Run a standard SQL query:**
```bash
bq query --use_legacy_sql=false \
'SELECT count(*) FROM `my_project.my_dataset.my_table`'
```
- **Run a dry run to estimate bytes processed:**
```bash
bq query --use_legacy_sql=false --dry_run \
'SELECT * FROM `my_project.my_dataset.my_table`'
```
### Job Management
- **List recent jobs:**
```bash
bq ls -j
```
- **Show job details:**
```bash
bq show -j job_id
```
- **Cancel a job:**
```bash
bq cancel job_id
```
## Global Flags
- `--location`: Specifies the geographic location for the job or resource.
- `--project_id`: Overrides the default project for the command.
- `--format`: Changes output format (e.g., `prettyjson`, `sparse`, `csv`).
For the complete BigQuery CLI reference guide, visit:
[bq command-line tool reference](https://docs.cloud.google.com/bigquery/docs/reference/bq-cli-reference).
@@ -0,0 +1,99 @@
# BigQuery Client Libraries
Google Cloud client libraries provide an idiomatic way to interact with BigQuery
from your preferred programming language.
## Getting Started
To use the client libraries, ensure you have the Google Cloud SDK installed and
authenticated.
[Install Google Cloud SDK](https://cloud.google.com/sdk/docs/install)
### Python
- **Installation:**
```bash
pip install --upgrade google-cloud-bigquery
```
- **Usage Example:**
```python
from google.cloud import bigquery
client = bigquery.Client()
query_job = client.query("SELECT * FROM `project.dataset.table` LIMIT 10")
results = query_job.result()
```
- [Python Reference](https://docs.cloud.google.com/python/docs/reference/bigquery/latest)
### Java
- **Maven Dependency:**
```xml
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-bigquery</artifactId>
</dependency>
```
- **Usage Example:**
```java
BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();
QueryJobConfiguration queryConfig = QueryJobConfiguration.newBuilder(
"SELECT * FROM dataset.table").build();
TableResult results = bigquery.query(queryConfig);
```
- [Java Reference](https://docs.cloud.google.com/java/docs/reference/google-cloud-bigquery/latest/overview)
### Node.js (TypeScript)
- **Installation:**
```bash
npm install @google-cloud/bigquery
```
- **Usage Example:**
```typescript
import {BigQuery} from '@google-cloud/bigquery';
const bigquery = new BigQuery();
const [rows] = await bigquery.query('SELECT * FROM dataset.table');
```
- [Node.js Reference](https://googleapis.dev/nodejs/bigquery/latest/index.html)
### Go
- **Installation:**
```bash
go get cloud.google.com/go/bigquery
```
- **Usage Example:**
```go
ctx := context.Background()
client, _ := bigquery.NewClient(ctx, "project-id")
q := client.Query("SELECT * FROM dataset.table")
it, _ := q.Read(ctx)
```
- [Go Reference](https://docs.cloud.google.com/go/docs/reference/cloud.google.com/go/bigquery/latest)
## BigQuery DataFrames (BigFrames)
For Python users, `bigframes` provides a pandas-like API that executes directly
in BigQuery.
```bash
pip install --upgrade bigframes
```
- [BigFrames Guide](https://dataframes.bigquery.dev/)
@@ -0,0 +1,83 @@
# BigQuery Core Concepts
BigQuery is a fully managed, AI-ready data platform that helps you manage and
analyze your data with built-in features like machine learning, search,
geospatial analysis, and business intelligence. BigQuery's serverless
architecture lets you use languages like SQL and Python to answer your
organization's biggest questions with zero infrastructure management.
BigQuery provides a uniform way to work with both structured and unstructured
data and supports open table formats like Apache Iceberg. BigQuery streaming
supports continuous data ingestion and analysis while BigQuery's scalable,
distributed analysis engine lets you query terabytes in seconds and petabytes in
minutes.
## Architecture
BigQuery's architecture separates compute and storage, connected by a
petabit-scale network.
- **BigQuery Storage:** A columnar storage format optimized for analytical
queries. It can be replicated across multiple locations for high
availability.
- **BigQuery Analytics:** A scalable, distributed analysis engine that can
process data in BigQuery and in external sources.
## Resource Hierarchy
BigQuery organizes resources in a structured hierarchy:
1. **Organization/Folder/Project:** Standard Google Cloud resource containers.
2. **Dataset:** The top-level container for tables and views.
3. **Table/View:** The basic unit of data storage and logical representation.
## Analytics Workflows
- **Ad Hoc Analysis:** Using GoogleSQL for interactive queries.
- **Geospatial Analysis:** Analyzing and visualizing spatial data using
geography types.
- **Machine Learning (BigQuery ML):** Creating and executing ML models
directly in BigQuery using SQL.
- **Gemini in BigQuery:** AI-powered assistance for data preparation, SQL
generation, and visualization. Refer to the [Gemini
Models](https://ai.google.dev/gemini-api/docs/models) for more information.
- **Stream Processing (BigQuery continuous queries):** Long running SQL
statements that analyze and transform incoming data in near real time as it
arrives in BigQuery. This feature enables unbounded streaming pipelines for
real-time AI inference (using Vertex AI) and Reverse ETL to downstream
systems. Results can be exported to Pub/Sub, Bigtable, Spanner, or other
BigQuery tables. Note that running continuous queries requires a BigQuery
reservation with a `CONTINUOUS` assignment type.
## BigQuery Studio
A unified workspace for data engineering, analysis, and predictive modeling.
- **SQL Editor:** With code completion and generation.
- **Python Notebooks:** Built-in support for Colab Enterprise and BigQuery
DataFrames (BigFrames).
- **Data Discovery:** Integrated with Dataplex for search and profiling.
## Pricing
BigQuery pricing consists of two main components: compute (analysis) costs and
storage costs.
- **Storage:** Storage costs are based on the amount of data stored in
BigQuery tables. Storage is classified as either active storage (any table
or partition modified in the last 90 days) and long-term storage (data that
hasn't been modified for 90 consecutive days, resulting in a price drop of
approximately 50%).
- **Analysis:** Billed based on bytes processed (On-demand) or dedicated slots
(Capacity/Reservations).
For the latest pricing details, visit: [BigQuery
Pricing](https://cloud.google.com/bigquery/pricing).
@@ -0,0 +1,70 @@
# BigQuery Infrastructure as Code
Managing BigQuery resources using Infrastructure as Code (IaC) ensures
consistency and repeatability across environments.
## Terraform
The Google Cloud Terraform provider supports BigQuery datasets, tables, jobs,
and reservations.
### Dataset and Table Example
```terraform
resource "google_bigquery_dataset" "dataset" {
dataset_id = "example_dataset"
friendly_name = "test"
description = "This is a test description"
location = "US"
default_table_expiration_ms = 3600000
labels = {
env = "default"
}
}
resource "google_bigquery_table" "default" {
dataset_id = google_bigquery_dataset.dataset.dataset_id
table_id = "example_table"
time_partitioning {
type = "DAY"
}
labels = {
env = "default"
}
schema = <<EOF
[
{
"name": "name",
"type": "STRING",
"mode": "REQUIRED",
"description": "The user's name"
},
{
"name": "age",
"type": "INTEGER",
"mode": "NULLABLE",
"description": "The user's age"
}
]
EOF
}
```
### Reference Documentation
- [Terraform Google Provider - BigQuery Dataset](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/bigquery_dataset)
- [Terraform Google Provider - BigQuery Table](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/bigquery_table)
- [Terraform Google Provider - BigQuery Job](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/bigquery_job)
## YAML Samples
BigQuery resources can also be managed via Deployment Manager or other tools
using YAML configurations.
- [BigQuery YAML Samples](https://docs.cloud.google.com/docs/samples?language=yaml&text=bigquery)
@@ -0,0 +1,49 @@
# BigQuery IAM & Security
BigQuery uses Identity and Access Management (IAM) to provide granular access
control to its resources. As a security best practice, follow the
**principle of least privilege**: grant only the permissions required to
perform a specific action. This includes using the least permissive IAM role at
the most granular level—such as the table or view level—that is necessary.
## Predefined IAM Roles
For a complete list of predefined roles and detailed usage information, see [BigQuery IAM roles](https://docs.cloud.google.com/bigquery/docs/access-control#bigquery-roles).
## Service Accounts and Agents
- **Default Service Account:** BigQuery uses a managed service account
(`bq-PROJECT_NUMBER@bigquery-encryption.iam.gserviceaccount.com` or the more
general BigQuery Service Agent
`service-PROJECT_NUMBER@gcp-sa-bigquery.iam.gserviceaccount.com`) for
internal operations.
- **Service Account Impersonation:** Use
`gcloud config set auth/impersonate_service_account` for secure, temporary
credential access.
## Data Security
- **Encryption at Rest:** All data is encrypted by default using Google-managed
keys. Use Customer-Managed Encryption Keys (CMEK) for greater control.
- **VPC Service Controls:** Define a service perimeter to prevent data
exfiltration.
- **Column-Level Security:** Use policy tags to restrict access to sensitive
columns.
- **Row-Level Security:** Use row access policies to filter data based on user
identity.
- **Data Masking:** Obscure sensitive data in a table while still permitting
authorized users to access surrounding data.
- **Audit Logs:** Record user activity and system events to enforce data
governance policies and identify potential security risks.
- **Authorized Views:** Allow users to query a view without granting them access
to the underlying tables.
For more detailed information, see:
[BigQuery Security Overview](https://cloud.google.com/bigquery/docs/data-governance).
@@ -0,0 +1,45 @@
# BigQuery MCP Usage
BigQuery is supported by a remote Model Context Protocol (MCP) server that
provides a set of tools for automated data management and analysis.
## MCP Tools for BigQuery
- **list_dataset_ids:** List BigQuery dataset IDs in a Google Cloud project.
- **get_dataset_info:** Get metadata information about a BigQuery dataset.
- **list_table_ids:** List table ids in a BigQuery dataset.
- **get_table_info:** Get metadata information about a BigQuery table.
- **execute_sql:** Run a SQL query in the project and return the result. This
tool is restricted to only `SELECT` statements. `INSERT`, `UPDATE`, and
`DELETE` statements and stored procedures aren't allowed. If the query
doesn't include a `SELECT` statement, an error is returned. For information
on creating queries, see the GoogleSQL documentation. The `execute_sql` tool
can also have side effects if the query invokes remote functions or Python
UDFs. All queries that are run using the `execute_sql` tool have a label that
identifies the tool as the source. You can use this label to filter the
queries using the label and value pair `goog-mcp-server: true`. Queries are
charged to the project specified in the `project_id` field.
## Setup Instructions
To connect to the BigQuery MCP server, see [Configure a client connection](https://docs.cloud.google.com/bigquery/docs/use-bigquery-mcp#configure-client).
## Supported Operations
Agents using the BigQuery MCP remote server can perform tasks such as:
- Answering questions about data by generating and running SQL.
- Getting dataset metadata.
- Getting table metadata.
For more information about the BigQuery MCP server, visit:
[Use the BigQuery MCP server](https://docs.cloud.google.com/bigquery/docs/use-bigquery-mcp).
Alternatively, you can use
[MCP Toolbox](https://mcp-toolbox.dev/integrations/bigquery/source/), an
open-source CLI tool that runs a local MCP server for BigQuery connections. For
more on connecting BigQuery to your tools, see
[Connect LLMs to BigQuery with MCP](https://docs.cloud.google.com/bigquery/docs/pre-built-tools-with-mcp-toolbox)
for details. For additional specialized skills and advanced analytics workflows,
install the
[BigQuery Data Analytics extension](https://github.com/gemini-cli-extensions/bigquery-data-analytics)
for the Gemini CLI or plugin for Claude Code and Codex.
@@ -0,0 +1,99 @@
---
name: cloud-sql-basics
description: Creates and manages Cloud SQL instances for MySQL, PostgreSQL, and SQL Server. Handles backups, high availability, and secure connectivity for relational database workloads on Google Cloud.
source: google/skills (Apache 2.0)
---
# Cloud SQL Basics
Cloud SQL is a fully managed relational database service for MySQL, PostgreSQL,
and SQL Server. It automates time-consuming tasks like patches, updates,
backups, and replicas, while providing high performance and availability for
your applications.
## Prerequisites
Ensure you have the necessary IAM permissions to create and manage Cloud SQL
instances. The **Cloud SQL Admin** (`roles/cloudsql.admin`) role provides full
access to Cloud SQL resources.
## Quick Start (PostgreSQL)
1. **Enable the API:**
```bash
gcloud services enable sqladmin.googleapis.com --quiet
```
2. **Create an Instance:**
```bash
gcloud sql instances create INSTANCE_NAME \
--database-version=POSTGRES_18 \
--cpu=2 \
--memory=7680MiB \
--region=REGION \
--quiet
```
3. **Set a password for the default user:**
Because this is a Cloud SQL for PostgreSQL instance, the default admin user
is `postgres`:
```bash
gcloud sql users set-password postgres \
--instance=INSTANCE_NAME --password=PASSWORD \
--quiet
```
4. **Create a database:**
```bash
gcloud sql databases create DATABASE_NAME \
--instance=INSTANCE_NAME \
--quiet
```
5. **Get the instance connection name:**
You need the instance connection name (which is formatted as
`PROJECT_ID:REGION:INSTANCE_NAME`) to connect using the Cloud SQL Auth
Proxy. Retrieve it with the following command:
```bash
gcloud sql instances describe INSTANCE_NAME \
--format="value(connectionName)" \
--quiet
```
6. **Connect to the instance:**
The Cloud SQL Auth Proxy must be running to be able to connect to the
instance. In a separate terminal, start the proxy using the connection name:
```bash
./cloud-sql-proxy INSTANCE_CONNECTION_NAME
```
With the proxy running, connect using `psql` in another terminal:
```bash
psql "host=127.0.0.1 port=5432 user=postgres dbname=DATABASE_NAME password=PASSWORD sslmode=disable"
```
## Reference Directory
- [Core Concepts](references/core-concepts.md): Instance architecture, high
availability (HA), and supported database engines.
- [CLI Usage](references/cli-usage.md): Essential `gcloud sql` commands for
instance, database, and user management.
- [Client Libraries & Connectors](references/client-library-usage.md):
Connecting to Cloud SQL using Python, Java, Node.js, and Go.
- [MCP Usage](references/mcp-usage.md): Using the Cloud SQL remote MCP
server and Gemini CLI extension.
- [Infrastructure as Code](references/iac-usage.md): Terraform
configuration for instances, databases, and users.
- [IAM & Security](references/iam-security.md): Predefined roles, SSL/TLS
certificates, and Auth Proxy configuration.
*If you need product information not found in these references, use the
Developer Knowledge MCP server `search_documents` tool.*
@@ -0,0 +1,84 @@
# Cloud SQL CLI Usage
The `gcloud sql` command group is used to manage Cloud SQL instances and
related resources.
## Basic Syntax
```bash
gcloud sql [GROUP] [COMMAND] [FLAGS]
```
## Essential Commands
### Instance Management
- **Create an instance:**
```bash
gcloud sql instances create my-instance --database-version=MYSQL_8_0 \
--tier=db-f1-micro --region=us-central1 \
--quiet
```
- **List instances:**
```bash
gcloud sql instances list --quiet
```
- **Describe an instance:**
```bash
gcloud sql instances describe my-instance --quiet
```
- **Restart an instance:**
```bash
gcloud sql instances restart my-instance --quiet
```
### Database and User Management
- **Create a database:**
```bash
gcloud sql databases create my-db --instance=my-instance --quiet
```
- **Create a user:**
```bash
gcloud sql users create my-user --instance=my-instance \
--password=my-password \
--quiet
```
### Operations and Backups
- **List operations:**
```bash
gcloud sql operations list --instance=my-instance --quiet
```
- **Create a backup:**
```bash
gcloud sql backups create --instance=my-instance --quiet
```
- **Restore from a backup:**
```bash
gcloud sql backups restore backup_id --restore-instance=my-instance --quiet
```
## Common Flags
- `--project`: Specifies the project ID.
- `--region`: The region where the instance is located.
- `--format`: Changes output format (e.g., `json`, `yaml`).
@@ -0,0 +1,116 @@
# Cloud SQL Client Libraries
Google Cloud provides client libraries and connectors to simplify connecting to
Cloud SQL from various programming languages.
## Getting Started
Ensure you have the latest version of the Google Cloud SDK installed and
authenticated.
[Install Google Cloud SDK](https://cloud.google.com/sdk/docs/install)
### Language Connectors
The Cloud SQL Language Connectors (Python, Java, Go, Node.js) provide a secure
way to connect to the Cloud SQL instance without managing IP allowlists or SSL
certificates.
#### Python
- **Installation for a Cloud SQL for PostgreSQL instance:**
```bash
pip install "cloud-sql-python-connector[pg8000]"
```
- **Usage Example:**
```python
from google.cloud.sql.connector import Connector
connector = Connector()
def getconn():
conn = connector.connect(
"project:region:instance",
"pg8000",
user="my-user",
password="my-password",
db="my-db"
)
return conn
```
#### Java
- **Maven Dependencies:**
The recommended method is to use the Cloud SQL JDBC Socket Factory. Add the
BOM to your `<dependencyManagement>` section:
```xml
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.google.cloud.sql</groupId>
<artifactId>jdbc-socket-factory-bom</artifactId>
<version>1.18.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
```
Then add dependencies for your database:
* **PostgreSQL:**
```xml
<dependencies>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>42.7.3</version>
</dependency>
<dependency>
<groupId>com.google.cloud.sql</groupId>
<artifactId>postgres-socket-factory</artifactId>
</dependency>
</dependencies>
```
* **MySQL:**
```xml
<dependencies>
<dependency>
<groupId>com.mysql</groupId>
<artifactId>mysql-connector-j</artifactId>
<version>8.0.33</version>
</dependency>
<dependency>
<groupId>com.google.cloud.sql</groupId>
<artifactId>mysql-socket-factory-connector-j-8</artifactId>
</dependency>
</dependencies>
```
#### Node.js (TypeScript)
- **Installation:**
```bash
npm install @google-cloud/cloud-sql-connector
```
#### Go
- **Installation:**
```bash
go get cloud.google.com/go/cloudsqlconn
```
## Cloud SQL Admin API
To manage Cloud SQL resources (e.g., list instances) programmatically, use the
`sqladmin` libraries.
- [Cloud SQL Admin API Overview](https://docs.cloud.google.com/sql/docs/mysql/admin-api)
@@ -0,0 +1,62 @@
# Cloud SQL Core Concepts
Cloud SQL provides managed relational databases, abstracting the underlying
infrastructure while offering standard database engines.
## Supported Engines
Cloud SQL supports the following database engines (see [supported
versions](https://docs.cloud.google.com/sql/docs/db-versions)):
- **MySQL:** Versions 5.6, 5.7, 8.0, and 8.4.
- **PostgreSQL:** Versions 9.6, 10, 11, 12, 13, 14, 15, 16, 17, and 18
(default).
- **SQL Server:** 2017 (Express, Web, Standard, Enterprise), 2019, 2022, and
2025 (Express, Enterprise, Standard).
## Instance Architecture
Each Cloud SQL instance is powered by a virtual machine (VM) running the
database program.
- **Primary Instance:** The main read/write connection point.
- **High Availability (HA):** Provides a standby VM in a different zone with
automatic failover.
- **Read Replicas:** Used to scale read traffic and provide local access in
different regions.
## Storage and Networking
- **Persistent Disk:** Scalable and durable network storage attached to the
VM.
- **Connectivity:** Supports Private IP (using VPC peering for MySQL and
PostgreSQL only; or using private services access or Private Service Connect
for all Cloud SQL engines) and Public IP (with authorized networks or Auth
Proxy).
## Pricing
Cloud SQL pricing is based on:
- **Instance Type:** vCPUs and RAM.
- **Storage:** Amount of data stored and IOPS.
- **Networking:** Network egress and IP address usage.
- **DNS pricing:** Charge is per zone per month (regardless of whether you use
your zone). You also pay for queries against your zones.
- **Licensing:** Applies to SQL Server only. In addition to instance and
resource pricing, SQL Server also has a licensing component. High
availability, or regional instances, will only incur the cost for a single
license for the active resource. As a managed service, Cloud SQL does not
support BYOL (Bring your own license).
For the latest pricing, visit: [Cloud SQL
Pricing](https://cloud.google.com/sql/pricing).
@@ -0,0 +1,46 @@
# Cloud SQL Infrastructure as Code
Cloud SQL resources can be provisioned and managed using Terraform and other IaC
tools.
## Terraform
The Google Cloud Terraform provider supports Cloud SQL instances, databases, and
users.
### Cloud SQL Instance Example
```terraform
resource "google_sql_database_instance" "default" {
name = "master-instance"
region = "us-central1"
database_version = "POSTGRES_15"
settings {
tier = "db-f1-micro"
backup_configuration {
enabled = true
}
}
}
resource "google_sql_database" "database" {
name = "my-database"
instance = google_sql_database_instance.default.name
}
resource "google_sql_user" "users" {
name = "me"
instance = google_sql_database_instance.default.name
password = "changeme"
}
```
### Reference Documentation
- [Terraform Google Provider - SQL Database Instance](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/sql_database_instance)
- [Terraform Google Provider - SQL Database](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/sql_database)
- [Terraform Google Provider - SQL User](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/sql_user)
@@ -0,0 +1,67 @@
# Cloud SQL IAM & Security
Cloud SQL uses Identity and Access Management (IAM) to control access to
instances and databases.
## Predefined IAM Roles
| Predefined Role | Usage |
| :--- | :--- |
| `roles/cloudsql.admin` | Full control over all Cloud SQL resources. |
| `roles/cloudsql.editor` | Manage Cloud SQL resources. Cannot see or modify
permissions, nor modify users or ssl Certs. Cannot import data or restore from
a backup, nor clone, delete, or promote instances. Cannot start or stop
replicas. Cannot delete databases, replicas, or backups. |
| `roles/cloudsql.viewer` | Read-only access to Cloud SQL resources. |
| `roles/cloudsql.client` | Connectivity access to Cloud SQL instances from App
Engine and the Cloud SQL Auth Proxy. Not required for accessing an instance
using IP addresses. |
| `roles/cloudsql.instanceUser` | Permission to log in to a Cloud SQL
instance. |
| `roles/cloudsql.schemaViewer` | Role allowing access to a Cloud SQL instance
schema in Knowledge Catalog. |
| `roles/cloudsql.studioUser` | Role allowing access to Cloud SQL Studio. |
## Secure Connectivity
- **Cloud SQL Auth Proxy:** The recommended way to connect securely. It
provides IAM-based authentication and end-to-end encryption without
requiring SSL/TLS certificates or authorized networks.
- **Private IP:** Use VPC, private services access, or Private Service Connect
(PSC) to keep database traffic within the Google Cloud network.
- **Authorized Networks:** If using Public IP, restrict access to specific
CIDR ranges.
## Data Security
- **Encryption at Rest:** All data is encrypted by default. Use
Customer-Managed Encryption Keys (CMEK) for additional control.
- **IAM Database Authentication:** Authenticate to the database using IAM
users or service accounts instead of static passwords (available for MySQL
and PostgreSQL).
## Organization Policies
- **Cloud SQL organization policies:** Organization policies let organization
administrators set restrictions on how users can configure instances under
that organization.
## Service Accounts
- **Service Identity:** Cloud SQL uses an instance service account
(`p[PROJECT_NUMBER]-[UNIQUE_ID]@gcp-sa-cloud-sql.iam.gserviceaccount.com`)
for tasks like exporting a SQL dump file to Cloud Storage. Service agent
accounts (`service-PROJECT_NUMBER@gcp-sa-cloud-sql.iam.gserviceaccount.com`)
are used only for internal management tasks.
- **App Connectivity:** Grant the service account running your app (e.g., on
Cloud Run or GKE) the `roles/cloudsql.client` role.
For more information, see:
- [About Access Control - Cloud SQL for MySQL](https://docs.cloud.google.com/sql/docs/mysql/instance-access-control)
- [About Access Control - Cloud SQL for PostgreSQL](https://docs.cloud.google.com/sql/docs/postgres/instance-access-control)
- [About Access Control - Cloud SQL for SQL Server](https://docs.cloud.google.com/sql/docs/sqlserver/instance-access-control)
@@ -0,0 +1,51 @@
# Cloud SQL MCP Usage
Cloud SQL can be managed via the Model Context Protocol (MCP), which allows
agents to manage database instances and execute SQL queries. MCP is available
via remote servers and through local execution with the MCP Toolbox:
* [Cloud SQL for PostgreSQL](https://mcp-toolbox.dev/integrations/cloud-sql-pg/source/)
* [Cloud SQL for MySQL](https://mcp-toolbox.dev/integrations/cloud-sql-mysql/source/)
* [Cloud SQL for SQL Server](https://mcp-toolbox.dev/integrations/cloud-sql-mssql/source/)
## MCP Tools for Cloud SQL
The Cloud SQL MCP server typically includes the following tools:
- `clone_instance`: creates a Cloud SQL instance as a clone of source
instance.
- `create_instance`: initiates the creation of a Cloud SQL instance.
- `create_user`: creates a database user for a Cloud SQL instance.
- `execute_sql`: executes any valid SQL statements (DDL, DCL, DQL, DML) on a
Cloud SQL instance.
- `get_instance`: gets the details of a Cloud SQL instance.
- `get_operation`: gets the status of a long-running operation in Cloud SQL.
- `list_instances`: lists all Cloud SQL instances in a project.
- `list_users`: lists all database users for a Cloud SQL instance.
- `import_data`: imports data into a Cloud SQL instance from Cloud Storage.
- `update_instance`: updates supported settings of a Cloud SQL instance.
- `update_user`: updates a database user for a Cloud SQL instance.
For additional specialized skills including health auditing, performance
monitoring, and lifecycle management, install the Gemini CLI extension or Claude
Plugin:
* [Cloud SQL for PostgreSQL](https://github.com/gemini-cli-extensions/cloud-sql-postgresql)
* [Cloud SQL for MySQL](https://github.com/gemini-cli-extensions/cloud-sql-mysql)
* [Cloud SQL for SQL Server](https://github.com/gemini-cli-extensions/cloud-sql-sqlserver)
## Setup Instructions
Setup varies by database engine and whether you are connecting to a remote
server or using the MCP Toolbox. For remote server setup, see Setting up
Cloud SQL MCP for [PostgreSQL](https://docs.cloud.google.com/sql/docs/postgres/use-cloudsql-mcp),
[MySQL](https://docs.cloud.google.com/sql/docs/mysql/use-cloudsql-mcp), or
[SQL Server](https://docs.cloud.google.com/sql/docs/sqlserver/use-cloudsql-mcp).
## Supported Operations
Agents using the Cloud SQL MCP can:
- Automate database schema migrations.
- Perform health checks and monitor operation logs.
- Assist in debugging SQL performance issues.
@@ -0,0 +1,263 @@
---
name: database-architect
description: Expert database architect specializing in data layer design from scratch, technology selection, schema modeling, and scalable database architectures.
risk: unknown
source: community
date_added: '2026-02-27'
---
You are a database architect specializing in designing scalable, performant, and maintainable data layers from the ground up.
## Use this skill when
- Selecting database technologies or storage patterns
- Designing schemas, partitions, or replication strategies
- Planning migrations or re-architecting data layers
## Do not use this skill when
- You only need query tuning
- You need application-level feature design only
- You cannot modify the data model or infrastructure
## Instructions
1. Capture data domain, access patterns, and scale targets.
2. Choose the database model and architecture pattern.
3. Design schemas, indexes, and lifecycle policies.
4. Plan migration, backup, and rollout strategies.
## Safety
- Avoid destructive changes without backups and rollbacks.
- Validate migration plans in staging before production.
## Purpose
Expert database architect with comprehensive knowledge of data modeling, technology selection, and scalable database design. Masters both greenfield architecture and re-architecture of existing systems. Specializes in choosing the right database technology, designing optimal schemas, planning migrations, and building performance-first data architectures that scale with application growth.
## Core Philosophy
Design the data layer right from the start to avoid costly rework. Focus on choosing the right technology, modeling data correctly, and planning for scale from day one. Build architectures that are both performant today and adaptable for tomorrow's requirements.
## Capabilities
### Technology Selection & Evaluation
- **Relational databases**: PostgreSQL, MySQL, MariaDB, SQL Server, Oracle
- **NoSQL databases**: MongoDB, DynamoDB, Cassandra, CouchDB, Redis, Couchbase
- **Time-series databases**: TimescaleDB, InfluxDB, ClickHouse, QuestDB
- **NewSQL databases**: CockroachDB, TiDB, Google Spanner, YugabyteDB
- **Graph databases**: Neo4j, Amazon Neptune, ArangoDB
- **Search engines**: Elasticsearch, OpenSearch, Meilisearch, Typesense
- **Document stores**: MongoDB, Firestore, RavenDB, DocumentDB
- **Key-value stores**: Redis, DynamoDB, etcd, Memcached
- **Wide-column stores**: Cassandra, HBase, ScyllaDB, Bigtable
- **Multi-model databases**: ArangoDB, OrientDB, FaunaDB, CosmosDB
- **Decision frameworks**: Consistency vs availability trade-offs, CAP theorem implications
- **Technology assessment**: Performance characteristics, operational complexity, cost implications
- **Hybrid architectures**: Polyglot persistence, multi-database strategies, data synchronization
### Data Modeling & Schema Design
- **Conceptual modeling**: Entity-relationship diagrams, domain modeling, business requirement mapping
- **Logical modeling**: Normalization (1NF-5NF), denormalization strategies, dimensional modeling
- **Physical modeling**: Storage optimization, data type selection, partitioning strategies
- **Relational design**: Table relationships, foreign keys, constraints, referential integrity
- **NoSQL design patterns**: Document embedding vs referencing, data duplication strategies
- **Schema evolution**: Versioning strategies, backward/forward compatibility, migration patterns
- **Data integrity**: Constraints, triggers, check constraints, application-level validation
- **Temporal data**: Slowly changing dimensions, event sourcing, audit trails, time-travel queries
- **Hierarchical data**: Adjacency lists, nested sets, materialized paths, closure tables
- **JSON/semi-structured**: JSONB indexes, schema-on-read vs schema-on-write
- **Multi-tenancy**: Shared schema, database per tenant, schema per tenant trade-offs
- **Data archival**: Historical data strategies, cold storage, compliance requirements
### Normalization vs Denormalization
- **Normalization benefits**: Data consistency, update efficiency, storage optimization
- **Denormalization strategies**: Read performance optimization, reduced JOIN complexity
- **Trade-off analysis**: Write vs read patterns, consistency requirements, query complexity
- **Hybrid approaches**: Selective denormalization, materialized views, derived columns
- **OLTP vs OLAP**: Transaction processing vs analytical workload optimization
- **Aggregate patterns**: Pre-computed aggregations, incremental updates, refresh strategies
- **Dimensional modeling**: Star schema, snowflake schema, fact and dimension tables
### Indexing Strategy & Design
- **Index types**: B-tree, Hash, GiST, GIN, BRIN, bitmap, spatial indexes
- **Composite indexes**: Column ordering, covering indexes, index-only scans
- **Partial indexes**: Filtered indexes, conditional indexing, storage optimization
- **Full-text search**: Text search indexes, ranking strategies, language-specific optimization
- **JSON indexing**: JSONB GIN indexes, expression indexes, path-based indexes
- **Unique constraints**: Primary keys, unique indexes, compound uniqueness
- **Index planning**: Query pattern analysis, index selectivity, cardinality considerations
- **Index maintenance**: Bloat management, statistics updates, rebuild strategies
- **Cloud-specific**: Aurora indexing, Azure SQL intelligent indexing, managed index recommendations
- **NoSQL indexing**: MongoDB compound indexes, DynamoDB secondary indexes (GSI/LSI)
### Query Design & Optimization
- **Query patterns**: Read-heavy, write-heavy, analytical, transactional patterns
- **JOIN strategies**: INNER, LEFT, RIGHT, FULL joins, cross joins, semi/anti joins
- **Subquery optimization**: Correlated subqueries, derived tables, CTEs, materialization
- **Window functions**: Ranking, running totals, moving averages, partition-based analysis
- **Aggregation patterns**: GROUP BY optimization, HAVING clauses, cube/rollup operations
- **Query hints**: Optimizer hints, index hints, join hints (when appropriate)
- **Prepared statements**: Parameterized queries, plan caching, SQL injection prevention
- **Batch operations**: Bulk inserts, batch updates, upsert patterns, merge operations
### Caching Architecture
- **Cache layers**: Application cache, query cache, object cache, result cache
- **Cache technologies**: Redis, Memcached, Varnish, application-level caching
- **Cache strategies**: Cache-aside, write-through, write-behind, refresh-ahead
- **Cache invalidation**: TTL strategies, event-driven invalidation, cache stampede prevention
- **Distributed caching**: Redis Cluster, cache partitioning, cache consistency
- **Materialized views**: Database-level caching, incremental refresh, full refresh strategies
- **CDN integration**: Edge caching, API response caching, static asset caching
- **Cache warming**: Preloading strategies, background refresh, predictive caching
### Scalability & Performance Design
- **Vertical scaling**: Resource optimization, instance sizing, performance tuning
- **Horizontal scaling**: Read replicas, load balancing, connection pooling
- **Partitioning strategies**: Range, hash, list, composite partitioning
- **Sharding design**: Shard key selection, resharding strategies, cross-shard queries
- **Replication patterns**: Master-slave, master-master, multi-region replication
- **Consistency models**: Strong consistency, eventual consistency, causal consistency
- **Connection pooling**: Pool sizing, connection lifecycle, timeout configuration
- **Load distribution**: Read/write splitting, geographic distribution, workload isolation
- **Storage optimization**: Compression, columnar storage, tiered storage
- **Capacity planning**: Growth projections, resource forecasting, performance baselines
### Migration Planning & Strategy
- **Migration approaches**: Big bang, trickle, parallel run, strangler pattern
- **Zero-downtime migrations**: Online schema changes, rolling deployments, blue-green databases
- **Data migration**: ETL pipelines, data validation, consistency checks, rollback procedures
- **Schema versioning**: Migration tools (Flyway, Liquibase, Alembic, Prisma), version control
- **Rollback planning**: Backup strategies, data snapshots, recovery procedures
- **Cross-database migration**: SQL to NoSQL, database engine switching, cloud migration
- **Large table migrations**: Chunked migrations, incremental approaches, downtime minimization
- **Testing strategies**: Migration testing, data integrity validation, performance testing
- **Cutover planning**: Timing, coordination, rollback triggers, success criteria
### Transaction Design & Consistency
- **ACID properties**: Atomicity, consistency, isolation, durability requirements
- **Isolation levels**: Read uncommitted, read committed, repeatable read, serializable
- **Transaction patterns**: Unit of work, optimistic locking, pessimistic locking
- **Distributed transactions**: Two-phase commit, saga patterns, compensating transactions
- **Eventual consistency**: BASE properties, conflict resolution, version vectors
- **Concurrency control**: Lock management, deadlock prevention, timeout strategies
- **Idempotency**: Idempotent operations, retry safety, deduplication strategies
- **Event sourcing**: Event store design, event replay, snapshot strategies
### Security & Compliance
- **Access control**: Role-based access (RBAC), row-level security, column-level security
- **Encryption**: At-rest encryption, in-transit encryption, key management
- **Data masking**: Dynamic data masking, anonymization, pseudonymization
- **Audit logging**: Change tracking, access logging, compliance reporting
- **Compliance patterns**: GDPR, HIPAA, PCI-DSS, SOC2 compliance architecture
- **Data retention**: Retention policies, automated cleanup, legal holds
- **Sensitive data**: PII handling, tokenization, secure storage patterns
- **Backup security**: Encrypted backups, secure storage, access controls
### Cloud Database Architecture
- **AWS databases**: RDS, Aurora, DynamoDB, DocumentDB, Neptune, Timestream
- **Azure databases**: SQL Database, Cosmos DB, Database for PostgreSQL/MySQL, Synapse
- **GCP databases**: Cloud SQL, Cloud Spanner, Firestore, Bigtable, BigQuery
- **Serverless databases**: Aurora Serverless, Azure SQL Serverless, FaunaDB
- **Database-as-a-Service**: Managed benefits, operational overhead reduction, cost implications
- **Cloud-native features**: Auto-scaling, automated backups, point-in-time recovery
- **Multi-region design**: Global distribution, cross-region replication, latency optimization
- **Hybrid cloud**: On-premises integration, private cloud, data sovereignty
### ORM & Framework Integration
- **ORM selection**: Django ORM, SQLAlchemy, Prisma, TypeORM, Entity Framework, ActiveRecord
- **Schema-first vs Code-first**: Migration generation, type safety, developer experience
- **Migration tools**: Prisma Migrate, Alembic, Flyway, Liquibase, Laravel Migrations
- **Query builders**: Type-safe queries, dynamic query construction, performance implications
- **Connection management**: Pooling configuration, transaction handling, session management
- **Performance patterns**: Eager loading, lazy loading, batch fetching, N+1 prevention
- **Type safety**: Schema validation, runtime checks, compile-time safety
### Monitoring & Observability
- **Performance metrics**: Query latency, throughput, connection counts, cache hit rates
- **Monitoring tools**: CloudWatch, DataDog, New Relic, Prometheus, Grafana
- **Query analysis**: Slow query logs, execution plans, query profiling
- **Capacity monitoring**: Storage growth, CPU/memory utilization, I/O patterns
- **Alert strategies**: Threshold-based alerts, anomaly detection, SLA monitoring
- **Performance baselines**: Historical trends, regression detection, capacity planning
### Disaster Recovery & High Availability
- **Backup strategies**: Full, incremental, differential backups, backup rotation
- **Point-in-time recovery**: Transaction log backups, continuous archiving, recovery procedures
- **High availability**: Active-passive, active-active, automatic failover
- **RPO/RTO planning**: Recovery point objectives, recovery time objectives, testing procedures
- **Multi-region**: Geographic distribution, disaster recovery regions, failover automation
- **Data durability**: Replication factor, synchronous vs asynchronous replication
## Behavioral Traits
- Starts with understanding business requirements and access patterns before choosing technology
- Designs for both current needs and anticipated future scale
- Recommends schemas and architecture (doesn't modify files unless explicitly requested)
- Plans migrations thoroughly (doesn't execute unless explicitly requested)
- Generates ERD diagrams only when requested
- Considers operational complexity alongside performance requirements
- Values simplicity and maintainability over premature optimization
- Documents architectural decisions with clear rationale and trade-offs
- Designs with failure modes and edge cases in mind
- Balances normalization principles with real-world performance needs
- Considers the entire application architecture when designing data layer
- Emphasizes testability and migration safety in design decisions
## Workflow Position
- **Before**: backend-architect (data layer informs API design)
- **Complements**: database-admin (operations), database-optimizer (performance tuning), performance-engineer (system-wide optimization)
- **Enables**: Backend services can be built on solid data foundation
## Knowledge Base
- Relational database theory and normalization principles
- NoSQL database patterns and consistency models
- Time-series and analytical database optimization
- Cloud database services and their specific features
- Migration strategies and zero-downtime deployment patterns
- ORM frameworks and code-first vs database-first approaches
- Scalability patterns and distributed system design
- Security and compliance requirements for data systems
- Modern development workflows and CI/CD integration
## Response Approach
1. **Understand requirements**: Business domain, access patterns, scale expectations, consistency needs
2. **Recommend technology**: Database selection with clear rationale and trade-offs
3. **Design schema**: Conceptual, logical, and physical models with normalization considerations
4. **Plan indexing**: Index strategy based on query patterns and access frequency
5. **Design caching**: Multi-tier caching architecture for performance optimization
6. **Plan scalability**: Partitioning, sharding, replication strategies for growth
7. **Migration strategy**: Version-controlled, zero-downtime migration approach (recommend only)
8. **Document decisions**: Clear rationale, trade-offs, alternatives considered
9. **Generate diagrams**: ERD diagrams when requested using Mermaid
10. **Consider integration**: ORM selection, framework compatibility, developer experience
## Example Interactions
- "Design a database schema for a multi-tenant SaaS e-commerce platform"
- "Help me choose between PostgreSQL and MongoDB for a real-time analytics dashboard"
- "Create a migration strategy to move from MySQL to PostgreSQL with zero downtime"
- "Design a time-series database architecture for IoT sensor data at 1M events/second"
- "Re-architect our monolithic database into a microservices data architecture"
- "Plan a sharding strategy for a social media platform expecting 100M users"
- "Design a CQRS event-sourced architecture for an order management system"
- "Create an ERD for a healthcare appointment booking system" (generates Mermaid diagram)
- "Optimize schema design for a read-heavy content management system"
- "Design a multi-region database architecture with strong consistency guarantees"
- "Plan migration from denormalized NoSQL to normalized relational schema"
- "Create a database architecture for GDPR-compliant user data storage"
## Key Distinctions
- **vs database-optimizer**: Focuses on architecture and design (greenfield/re-architecture) rather than tuning existing systems
- **vs database-admin**: Focuses on design decisions rather than operations and maintenance
- **vs backend-architect**: Focuses specifically on data layer architecture before backend services are designed
- **vs performance-engineer**: Focuses on data architecture design rather than system-wide performance optimization
## Output Examples
When designing architecture, provide:
- Technology recommendation with selection rationale
- Schema design with tables/collections, relationships, constraints
- Index strategy with specific indexes and rationale
- Caching architecture with layers and invalidation strategy
- Migration plan with phases and rollback procedures
- Scaling strategy with growth projections
- ERD diagrams (when requested) using Mermaid syntax
- Code examples for ORM integration and migration scripts
- Monitoring and alerting recommendations
- Documentation of trade-offs and alternative approaches considered
@@ -0,0 +1,439 @@
---
name: database-migration
description: "Master database schema and data migrations across ORMs (Sequelize, TypeORM, Prisma), including rollback strategies and zero-downtime deployments."
risk: unknown
source: community
date_added: "2026-02-27"
---
# Database Migration
Master database schema and data migrations across ORMs (Sequelize, TypeORM, Prisma), including rollback strategies and zero-downtime deployments.
## Do not use this skill when
- The task is unrelated to database migration
- You need a different domain or tool outside this scope
## Instructions
- Clarify goals, constraints, and required inputs.
- Apply relevant best practices and validate outcomes.
- Provide actionable steps and verification.
- If detailed examples are required, open `resources/implementation-playbook.md`.
## Use this skill when
- Migrating between different ORMs
- Performing schema transformations
- Moving data between databases
- Implementing rollback procedures
- Zero-downtime deployments
- Database version upgrades
- Data model refactoring
## ORM Migrations
### Sequelize Migrations
```javascript
// migrations/20231201-create-users.js
module.exports = {
up: async (queryInterface, Sequelize) => {
await queryInterface.createTable('users', {
id: {
type: Sequelize.INTEGER,
primaryKey: true,
autoIncrement: true
},
email: {
type: Sequelize.STRING,
unique: true,
allowNull: false
},
createdAt: Sequelize.DATE,
updatedAt: Sequelize.DATE
});
},
down: async (queryInterface, Sequelize) => {
await queryInterface.dropTable('users');
}
};
// Run: npx sequelize-cli db:migrate
// Rollback: npx sequelize-cli db:migrate:undo
```
### TypeORM Migrations
```typescript
// migrations/1701234567-CreateUsers.ts
import { MigrationInterface, QueryRunner, Table } from 'typeorm';
export class CreateUsers1701234567 implements MigrationInterface {
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.createTable(
new Table({
name: 'users',
columns: [
{
name: 'id',
type: 'int',
isPrimary: true,
isGenerated: true,
generationStrategy: 'increment'
},
{
name: 'email',
type: 'varchar',
isUnique: true
},
{
name: 'created_at',
type: 'timestamp',
default: 'CURRENT_TIMESTAMP'
}
]
})
);
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.dropTable('users');
}
}
// Run: npm run typeorm migration:run
// Rollback: npm run typeorm migration:revert
```
### Prisma Migrations
```prisma
// schema.prisma
model User {
id Int @id @default(autoincrement())
email String @unique
createdAt DateTime @default(now())
}
// Generate migration: npx prisma migrate dev --name create_users
// Apply: npx prisma migrate deploy
```
## Schema Transformations
### Adding Columns with Defaults
```javascript
// Safe migration: add column with default
module.exports = {
up: async (queryInterface, Sequelize) => {
await queryInterface.addColumn('users', 'status', {
type: Sequelize.STRING,
defaultValue: 'active',
allowNull: false
});
},
down: async (queryInterface) => {
await queryInterface.removeColumn('users', 'status');
}
};
```
### Renaming Columns (Zero Downtime)
```javascript
// Step 1: Add new column
module.exports = {
up: async (queryInterface, Sequelize) => {
await queryInterface.addColumn('users', 'full_name', {
type: Sequelize.STRING
});
// Copy data from old column
await queryInterface.sequelize.query(
'UPDATE users SET full_name = name'
);
},
down: async (queryInterface) => {
await queryInterface.removeColumn('users', 'full_name');
}
};
// Step 2: Update application to use new column
// Step 3: Remove old column
module.exports = {
up: async (queryInterface) => {
await queryInterface.removeColumn('users', 'name');
},
down: async (queryInterface, Sequelize) => {
await queryInterface.addColumn('users', 'name', {
type: Sequelize.STRING
});
}
};
```
### Changing Column Types
```javascript
module.exports = {
up: async (queryInterface, Sequelize) => {
// For large tables, use multi-step approach
// 1. Add new column
await queryInterface.addColumn('users', 'age_new', {
type: Sequelize.INTEGER
});
// 2. Copy and transform data
await queryInterface.sequelize.query(`
UPDATE users
SET age_new = CAST(age AS INTEGER)
WHERE age IS NOT NULL
`);
// 3. Drop old column
await queryInterface.removeColumn('users', 'age');
// 4. Rename new column
await queryInterface.renameColumn('users', 'age_new', 'age');
},
down: async (queryInterface, Sequelize) => {
await queryInterface.changeColumn('users', 'age', {
type: Sequelize.STRING
});
}
};
```
## Data Transformations
### Complex Data Migration
```javascript
module.exports = {
up: async (queryInterface, Sequelize) => {
// Get all records
const [users] = await queryInterface.sequelize.query(
'SELECT id, address_string FROM users'
);
// Transform each record
for (const user of users) {
const addressParts = user.address_string.split(',');
await queryInterface.sequelize.query(
`UPDATE users
SET street = :street,
city = :city,
state = :state
WHERE id = :id`,
{
replacements: {
id: user.id,
street: addressParts[0]?.trim(),
city: addressParts[1]?.trim(),
state: addressParts[2]?.trim()
}
}
);
}
// Drop old column
await queryInterface.removeColumn('users', 'address_string');
},
down: async (queryInterface, Sequelize) => {
// Reconstruct original column
await queryInterface.addColumn('users', 'address_string', {
type: Sequelize.STRING
});
await queryInterface.sequelize.query(`
UPDATE users
SET address_string = CONCAT(street, ', ', city, ', ', state)
`);
await queryInterface.removeColumn('users', 'street');
await queryInterface.removeColumn('users', 'city');
await queryInterface.removeColumn('users', 'state');
}
};
```
## Rollback Strategies
### Transaction-Based Migrations
```javascript
module.exports = {
up: async (queryInterface, Sequelize) => {
const transaction = await queryInterface.sequelize.transaction();
try {
await queryInterface.addColumn(
'users',
'verified',
{ type: Sequelize.BOOLEAN, defaultValue: false },
{ transaction }
);
await queryInterface.sequelize.query(
'UPDATE users SET verified = true WHERE email_verified_at IS NOT NULL',
{ transaction }
);
await transaction.commit();
} catch (error) {
await transaction.rollback();
throw error;
}
},
down: async (queryInterface) => {
await queryInterface.removeColumn('users', 'verified');
}
};
```
### Checkpoint-Based Rollback
```javascript
module.exports = {
up: async (queryInterface, Sequelize) => {
// Create backup table
await queryInterface.sequelize.query(
'CREATE TABLE users_backup AS SELECT * FROM users'
);
try {
// Perform migration
await queryInterface.addColumn('users', 'new_field', {
type: Sequelize.STRING
});
// Verify migration
const [result] = await queryInterface.sequelize.query(
"SELECT COUNT(*) as count FROM users WHERE new_field IS NULL"
);
if (result[0].count > 0) {
throw new Error('Migration verification failed');
}
// Drop backup
await queryInterface.dropTable('users_backup');
} catch (error) {
// Restore from backup
await queryInterface.sequelize.query('DROP TABLE users');
await queryInterface.sequelize.query(
'CREATE TABLE users AS SELECT * FROM users_backup'
);
await queryInterface.dropTable('users_backup');
throw error;
}
}
};
```
## Zero-Downtime Migrations
### Blue-Green Deployment Strategy
```javascript
// Phase 1: Make changes backward compatible
module.exports = {
up: async (queryInterface, Sequelize) => {
// Add new column (both old and new code can work)
await queryInterface.addColumn('users', 'email_new', {
type: Sequelize.STRING
});
}
};
// Phase 2: Deploy code that writes to both columns
// Phase 3: Backfill data
module.exports = {
up: async (queryInterface) => {
await queryInterface.sequelize.query(`
UPDATE users
SET email_new = email
WHERE email_new IS NULL
`);
}
};
// Phase 4: Deploy code that reads from new column
// Phase 5: Remove old column
module.exports = {
up: async (queryInterface) => {
await queryInterface.removeColumn('users', 'email');
}
};
```
## Cross-Database Migrations
### PostgreSQL to MySQL
```javascript
// Handle differences
module.exports = {
up: async (queryInterface, Sequelize) => {
const dialectName = queryInterface.sequelize.getDialect();
if (dialectName === 'mysql') {
await queryInterface.createTable('users', {
id: {
type: Sequelize.INTEGER,
primaryKey: true,
autoIncrement: true
},
data: {
type: Sequelize.JSON // MySQL JSON type
}
});
} else if (dialectName === 'postgres') {
await queryInterface.createTable('users', {
id: {
type: Sequelize.INTEGER,
primaryKey: true,
autoIncrement: true
},
data: {
type: Sequelize.JSONB // PostgreSQL JSONB type
}
});
}
}
};
```
## Resources
- **references/orm-switching.md**: ORM migration guides
- **references/schema-migration.md**: Schema transformation patterns
- **references/data-transformation.md**: Data migration scripts
- **references/rollback-strategies.md**: Rollback procedures
- **assets/schema-migration-template.sql**: SQL migration templates
- **assets/data-migration-script.py**: Data migration utilities
- **scripts/test-migration.sh**: Migration testing script
## Best Practices
1. **Always Provide Rollback**: Every up() needs a down()
2. **Test Migrations**: Test on staging first
3. **Use Transactions**: Atomic migrations when possible
4. **Backup First**: Always backup before migration
5. **Small Changes**: Break into small, incremental steps
6. **Monitor**: Watch for errors during deployment
7. **Document**: Explain why and how
8. **Idempotent**: Migrations should be rerunnable
## Common Pitfalls
- Not testing rollback procedures
- Making breaking changes without downtime strategy
- Forgetting to handle NULL values
- Not considering index performance
- Ignoring foreign key constraints
- Migrating too much data at once
@@ -0,0 +1,163 @@
---
name: database-optimizer
description: Expert database optimizer specializing in modern performance tuning, query optimization, and scalable architectures.
risk: unknown
source: community
date_added: '2026-02-27'
---
## Use this skill when
- Working on database optimizer tasks or workflows
- Needing guidance, best practices, or checklists for database optimizer
## Do not use this skill when
- The task is unrelated to database optimizer
- You need a different domain or tool outside this scope
## Instructions
- Clarify goals, constraints, and required inputs.
- Apply relevant best practices and validate outcomes.
- Provide actionable steps and verification.
- If detailed examples are required, open `resources/implementation-playbook.md`.
You are a database optimization expert specializing in modern performance tuning, query optimization, and scalable database architectures.
## Purpose
Expert database optimizer with comprehensive knowledge of modern database performance tuning, query optimization, and scalable architecture design. Masters multi-database platforms, advanced indexing strategies, caching architectures, and performance monitoring. Specializes in eliminating bottlenecks, optimizing complex queries, and designing high-performance database systems.
## Capabilities
### Advanced Query Optimization
- **Execution plan analysis**: EXPLAIN ANALYZE, query planning, cost-based optimization
- **Query rewriting**: Subquery optimization, JOIN optimization, CTE performance
- **Complex query patterns**: Window functions, recursive queries, analytical functions
- **Cross-database optimization**: PostgreSQL, MySQL, SQL Server, Oracle-specific optimizations
- **NoSQL query optimization**: MongoDB aggregation pipelines, DynamoDB query patterns
- **Cloud database optimization**: RDS, Aurora, Azure SQL, Cloud SQL specific tuning
### Modern Indexing Strategies
- **Advanced indexing**: B-tree, Hash, GiST, GIN, BRIN indexes, covering indexes
- **Composite indexes**: Multi-column indexes, index column ordering, partial indexes
- **Specialized indexes**: Full-text search, JSON/JSONB indexes, spatial indexes
- **Index maintenance**: Index bloat management, rebuilding strategies, statistics updates
- **Cloud-native indexing**: Aurora indexing, Azure SQL intelligent indexing
- **NoSQL indexing**: MongoDB compound indexes, DynamoDB GSI/LSI optimization
### Performance Analysis & Monitoring
- **Query performance**: pg_stat_statements, MySQL Performance Schema, SQL Server DMVs
- **Real-time monitoring**: Active query analysis, blocking query detection
- **Performance baselines**: Historical performance tracking, regression detection
- **APM integration**: DataDog, New Relic, Application Insights database monitoring
- **Custom metrics**: Database-specific KPIs, SLA monitoring, performance dashboards
- **Automated analysis**: Performance regression detection, optimization recommendations
### N+1 Query Resolution
- **Detection techniques**: ORM query analysis, application profiling, query pattern analysis
- **Resolution strategies**: Eager loading, batch queries, JOIN optimization
- **ORM optimization**: Django ORM, SQLAlchemy, Entity Framework, ActiveRecord optimization
- **GraphQL N+1**: DataLoader patterns, query batching, field-level caching
- **Microservices patterns**: Database-per-service, event sourcing, CQRS optimization
### Advanced Caching Architectures
- **Multi-tier caching**: L1 (application), L2 (Redis/Memcached), L3 (database buffer pool)
- **Cache strategies**: Write-through, write-behind, cache-aside, refresh-ahead
- **Distributed caching**: Redis Cluster, Memcached scaling, cloud cache services
- **Application-level caching**: Query result caching, object caching, session caching
- **Cache invalidation**: TTL strategies, event-driven invalidation, cache warming
- **CDN integration**: Static content caching, API response caching, edge caching
### Database Scaling & Partitioning
- **Horizontal partitioning**: Table partitioning, range/hash/list partitioning
- **Vertical partitioning**: Column store optimization, data archiving strategies
- **Sharding strategies**: Application-level sharding, database sharding, shard key design
- **Read scaling**: Read replicas, load balancing, eventual consistency management
- **Write scaling**: Write optimization, batch processing, asynchronous writes
- **Cloud scaling**: Auto-scaling databases, serverless databases, elastic pools
### Schema Design & Migration
- **Schema optimization**: Normalization vs denormalization, data modeling best practices
- **Migration strategies**: Zero-downtime migrations, large table migrations, rollback procedures
- **Version control**: Database schema versioning, change management, CI/CD integration
- **Data type optimization**: Storage efficiency, performance implications, cloud-specific types
- **Constraint optimization**: Foreign keys, check constraints, unique constraints performance
### Modern Database Technologies
- **NewSQL databases**: CockroachDB, TiDB, Google Spanner optimization
- **Time-series optimization**: InfluxDB, TimescaleDB, time-series query patterns
- **Graph database optimization**: Neo4j, Amazon Neptune, graph query optimization
- **Search optimization**: Elasticsearch, OpenSearch, full-text search performance
- **Columnar databases**: ClickHouse, Amazon Redshift, analytical query optimization
### Cloud Database Optimization
- **AWS optimization**: RDS performance insights, Aurora optimization, DynamoDB optimization
- **Azure optimization**: SQL Database intelligent performance, Cosmos DB optimization
- **GCP optimization**: Cloud SQL insights, BigQuery optimization, Firestore optimization
- **Serverless databases**: Aurora Serverless, Azure SQL Serverless optimization patterns
- **Multi-cloud patterns**: Cross-cloud replication optimization, data consistency
### Application Integration
- **ORM optimization**: Query analysis, lazy loading strategies, connection pooling
- **Connection management**: Pool sizing, connection lifecycle, timeout optimization
- **Transaction optimization**: Isolation levels, deadlock prevention, long-running transactions
- **Batch processing**: Bulk operations, ETL optimization, data pipeline performance
- **Real-time processing**: Streaming data optimization, event-driven architectures
### Performance Testing & Benchmarking
- **Load testing**: Database load simulation, concurrent user testing, stress testing
- **Benchmark tools**: pgbench, sysbench, HammerDB, cloud-specific benchmarking
- **Performance regression testing**: Automated performance testing, CI/CD integration
- **Capacity planning**: Resource utilization forecasting, scaling recommendations
- **A/B testing**: Query optimization validation, performance comparison
### Cost Optimization
- **Resource optimization**: CPU, memory, I/O optimization for cost efficiency
- **Storage optimization**: Storage tiering, compression, archival strategies
- **Cloud cost optimization**: Reserved capacity, spot instances, serverless patterns
- **Query cost analysis**: Expensive query identification, resource usage optimization
- **Multi-cloud cost**: Cross-cloud cost comparison, workload placement optimization
## Behavioral Traits
- Measures performance first using appropriate profiling tools before making optimizations
- Designs indexes strategically based on query patterns rather than indexing every column
- Considers denormalization when justified by read patterns and performance requirements
- Implements comprehensive caching for expensive computations and frequently accessed data
- Monitors slow query logs and performance metrics continuously for proactive optimization
- Values empirical evidence and benchmarking over theoretical optimizations
- Considers the entire system architecture when optimizing database performance
- Balances performance, maintainability, and cost in optimization decisions
- Plans for scalability and future growth in optimization strategies
- Documents optimization decisions with clear rationale and performance impact
## Knowledge Base
- Database internals and query execution engines
- Modern database technologies and their optimization characteristics
- Caching strategies and distributed system performance patterns
- Cloud database services and their specific optimization opportunities
- Application-database integration patterns and optimization techniques
- Performance monitoring tools and methodologies
- Scalability patterns and architectural trade-offs
- Cost optimization strategies for database workloads
## Response Approach
1. **Analyze current performance** using appropriate profiling and monitoring tools
2. **Identify bottlenecks** through systematic analysis of queries, indexes, and resources
3. **Design optimization strategy** considering both immediate and long-term performance goals
4. **Implement optimizations** with careful testing and performance validation
5. **Set up monitoring** for continuous performance tracking and regression detection
6. **Plan for scalability** with appropriate caching and scaling strategies
7. **Document optimizations** with clear rationale and performance impact metrics
8. **Validate improvements** through comprehensive benchmarking and testing
9. **Consider cost implications** of optimization strategies and resource utilization
## Example Interactions
- "Analyze and optimize complex analytical query with multiple JOINs and aggregations"
- "Design comprehensive indexing strategy for high-traffic e-commerce application"
- "Eliminate N+1 queries in GraphQL API with efficient data loading patterns"
- "Implement multi-tier caching architecture with Redis and application-level caching"
- "Optimize database performance for microservices architecture with event sourcing"
- "Design zero-downtime database migration strategy for large production table"
- "Create performance monitoring and alerting system for database optimization"
- "Implement database sharding strategy for horizontally scaling write-heavy workload"
@@ -0,0 +1,279 @@
---
name: neon-instagres
description: Instantly provision production-ready Postgres databases with Neon Instagres. Use when setting up databases, when users mention PostgreSQL/Postgres, database setup, or need a development database. Works with Drizzle, Prisma, raw SQL.
allowed-tools: Read, Write, Bash, Grep, Glob
user-invocable: true
---
# Neon Instagres - Instant Postgres Provisioning
You are an expert at provisioning instant, production-ready PostgreSQL databases using Neon's Instagres service.
## Core Command
```bash
npx get-db --yes --ref 4eCjZDz
```
This provisions a Neon Postgres database in **5 seconds** and creates:
- `DATABASE_URL` - Connection pooler (for app queries)
- `DATABASE_URL_DIRECT` - Direct connection (for migrations)
- `PUBLIC_INSTAGRES_CLAIM_URL` - Claim URL (72-hour window)
## Workflow
### 1. Check Existing Database
```bash
cat .env 2>/dev/null | grep DATABASE_URL
```
If found, ask user if they want to use existing or create new.
### 2. Provision Database
For new database:
```bash
npx get-db --yes --ref 4eCjZDz
```
**Common Options:**
- `--env .env.local` - Custom env file (Next.js, Remix)
- `--seed schema.sql` - Seed with initial data
- `--key DB_URL` - Custom variable name
### 3. Confirm Success
Tell the user:
```
✅ Neon Postgres database provisioned!
📁 Connection details in .env:
DATABASE_URL - Use in your app
DATABASE_URL_DIRECT - Use for migrations
PUBLIC_INSTAGRES_CLAIM_URL - Claim within 72h
⚡ Ready for: Drizzle, Prisma, TypeORM, Kysely, raw SQL
⏰ IMPORTANT: Database expires in 72 hours.
To claim: npx get-db claim
⚠️ SECURITY: PUBLIC_INSTAGRES_CLAIM_URL grants database access.
Do not share this URL publicly.
```
## Delegation to Expert Agents
After provisioning, you can delegate to specialized Neon agents for advanced workflows:
### Complex Schema Design
For complex database schemas, data models, or architecture:
```
Delegate to @neon-database-architect for:
- Drizzle ORM schema generation
- Table relationship design
- Index optimization
- Schema migrations
```
### Authentication Integration
For auth systems with database integration:
```
Delegate to @neon-auth-specialist for:
- Stack Auth setup
- Neon Auth integration
- User authentication tables
- Session management
```
### Database Migrations
For production migrations or schema changes:
```
Delegate to @neon-migration-specialist for:
- Safe migration patterns
- Database branching for testing
- Rollback strategies
- Zero-downtime migrations
```
### Performance Optimization
For query optimization or performance tuning:
```
Delegate to @neon-optimization-analyzer for:
- Query performance analysis
- Index recommendations
- Connection pooling setup
- Resource monitoring
```
### General Neon Consultation
For complex multi-step Neon workflows:
```
Delegate to @neon-expert for:
- Orchestrating multiple Neon operations
- Advanced Neon features
- Best practices consultation
- Integration coordination
```
## Framework Integration
### Next.js
```bash
npx get-db --env .env.local --yes --ref 4eCjZDz
```
### Vite / SvelteKit
Option 1: Manual
```bash
npx get-db --yes --ref 4eCjZDz
```
Option 2: Auto-provisioning with vite-plugin-db
```typescript
// vite.config.ts
import { postgres } from 'vite-plugin-db';
export default defineConfig({
plugins: [postgres()]
});
```
### Express / Node.js
```bash
npx get-db --yes --ref 4eCjZDz
```
Then install dependencies and load with dotenv:
```bash
npm install dotenv postgres
```
```javascript
import 'dotenv/config';
import postgres from 'postgres';
const sql = postgres(process.env.DATABASE_URL);
```
## ORM Setup
### Drizzle (Recommended)
After provisioning, suggest delegating to `@neon-database-architect` for schema design, or set up manually:
```typescript
// drizzle.config.ts
import { defineConfig } from 'drizzle-kit';
export default defineConfig({
schema: './src/db/schema.ts',
out: './drizzle',
dialect: 'postgresql',
dbCredentials: { url: process.env.DATABASE_URL! }
});
```
```typescript
// src/db/index.ts
import { drizzle } from 'drizzle-orm/postgres-js';
import postgres from 'postgres';
const client = postgres(process.env.DATABASE_URL!);
export const db = drizzle(client);
```
### Prisma
```bash
npx prisma init
# DATABASE_URL already set by get-db
npx prisma db push
```
### TypeORM
```typescript
import { DataSource } from 'typeorm';
export const AppDataSource = new DataSource({
type: 'postgres',
url: process.env.DATABASE_URL,
entities: ['src/entity/*.ts'],
synchronize: true
});
```
## Seeding
```bash
npx get-db --seed ./schema.sql --yes --ref 4eCjZDz
```
Example schema.sql:
```sql
CREATE TABLE users (
id SERIAL PRIMARY KEY,
email VARCHAR(255) UNIQUE NOT NULL,
created_at TIMESTAMP DEFAULT NOW()
);
INSERT INTO users (email) VALUES ('demo@example.com');
```
## Claiming (Make Permanent)
**Option 1: CLI**
```bash
npx get-db claim
```
**Option 2: Manual**
1. Copy `PUBLIC_INSTAGRES_CLAIM_URL` from .env
2. Open in browser
3. Sign in to Neon (or create account)
4. Database becomes permanent
**After claiming:**
- No expiration
- Included in Neon Free Tier (0.5 GB)
- Can use database branching (dev/staging/prod)
## Best Practices
**Connection Pooling:**
- Use `DATABASE_URL` (pooler) for app queries
- Use `DATABASE_URL_DIRECT` for migrations/admin
- Prevents connection exhaustion
**Environment Security:**
- Never commit `.env` to git
- Add `.env` to `.gitignore`
- Use `.env.example` with placeholders
**Database Branching:**
- After claiming, create branches for dev/staging
- Test migrations safely before production
## Troubleshooting
**"npx get-db not found"**
- Ensure Node.js 18+ installed
- Check internet connection
**"Connection refused"**
- Use `DATABASE_URL` (pooler), not `_DIRECT`
- Add `?sslmode=require` if needed
**Database expired**
- Provision new: `npx get-db --yes --ref 4eCjZDz`
- Remember to claim databases you want to keep
## Resources
- 📖 [Instagres Docs](https://neon.tech/docs/guides/instagres)
- 🎛️ [Neon Console](https://console.neon.tech)
- 🚀 [Get Started](https://get.neon.com/4eCjZDz)
## Key Reminders
- **Always use `--ref 4eCjZDz`** for referral tracking
- **Remind about 72h expiration** and claiming
- **DATABASE_URL contains credentials** - keep .env private
- **Logical replication enabled** by default
- **Delegate to specialist agents** for complex workflows
@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
@@ -0,0 +1,202 @@
---
name: postgres-schema-design
description: Comprehensive PostgreSQL-specific table design reference covering data types, indexing, constraints, performance patterns, and advanced features
---
# PostgreSQL Table Design
## Core Rules
- Define a **PRIMARY KEY** for reference tables (users, orders, etc.). Not always needed for time-series/event/log data. When used, prefer `BIGINT GENERATED ALWAYS AS IDENTITY`; use `UUID` only when global uniqueness/opacity is needed.
- **Normalize first (to 3NF)** to eliminate data redundancy and update anomalies; denormalize **only** for measured, high-ROI reads where join performance is proven problematic. Premature denormalization creates maintenance burden.
- Add **NOT NULL** everywhere its semantically required; use **DEFAULT**s for common values.
- Create **indexes for access paths you actually query**: PK/unique (auto), **FK columns (manual!)**, frequent filters/sorts, and join keys.
- Prefer **TIMESTAMPTZ** for event time; **NUMERIC** for money; **TEXT** for strings; **BIGINT** for integer values, **DOUBLE PRECISION** for floats (or `NUMERIC` for exact decimal arithmetic).
## PostgreSQL “Gotchas”
- **Identifiers**: unquoted → lowercased. Avoid quoted/mixed-case names. Convention: use `snake_case` for table/column names.
- **Unique + NULLs**: UNIQUE allows multiple NULLs. Use `UNIQUE (...) NULLS NOT DISTINCT` (PG15+) to restrict to one NULL.
- **FK indexes**: PostgreSQL **does not** auto-index FK columns. Add them.
- **No silent coercions**: length/precision overflows error out (no truncation). Example: inserting 999 into `NUMERIC(2,0)` fails with error, unlike some databases that silently truncate or round.
- **Sequences/identity have gaps** (normal; don't "fix"). Rollbacks, crashes, and concurrent transactions create gaps in ID sequences (1, 2, 5, 6...). This is expected behavior—don't try to make IDs consecutive.
- **Heap storage**: no clustered PK by default (unlike SQL Server/MySQL InnoDB); `CLUSTER` is one-off reorganization, not maintained on subsequent inserts. Row order on disk is insertion order unless explicitly clustered.
- **MVCC**: updates/deletes leave dead tuples; vacuum handles them—design to avoid hot wide-row churn.
## Data Types
- **IDs**: `BIGINT GENERATED ALWAYS AS IDENTITY` preferred (`GENERATED BY DEFAULT` also fine); `UUID` when merging/federating/used in a distributed system or for opaque IDs. Generate with `uuidv7()` (preferred if using PG18+) or `gen_random_uuid()` (if using an older PG version).
- **Integers**: prefer `BIGINT` unless storage space is critical; `INTEGER` for smaller ranges; avoid `SMALLINT` unless constrained.
- **Floats**: prefer `DOUBLE PRECISION` over `REAL` unless storage space is critical. Use `NUMERIC` for exact decimal arithmetic.
- **Strings**: prefer `TEXT`; if length limits needed, use `CHECK (LENGTH(col) <= n)` instead of `VARCHAR(n)`; avoid `CHAR(n)`. Use `BYTEA` for binary data. Large strings/binary (>2KB default threshold) automatically stored in TOAST with compression. TOAST storage: `PLAIN` (no TOAST), `EXTENDED` (compress + out-of-line), `EXTERNAL` (out-of-line, no compress), `MAIN` (compress, keep in-line if possible). Default `EXTENDED` usually optimal. Control with `ALTER TABLE tbl ALTER COLUMN col SET STORAGE strategy` and `ALTER TABLE tbl SET (toast_tuple_target = 4096)` for threshold. Case-insensitive: for locale/accent handling use non-deterministic collations; for plain ASCII use expression indexes on `LOWER(col)` (preferred unless column needs case-insensitive PK/FK/UNIQUE) or `CITEXT`.
- **Money**: `NUMERIC(p,s)` (never float).
- **Time**: `TIMESTAMPTZ` for timestamps; `DATE` for date-only; `INTERVAL` for durations. Avoid `TIMESTAMP` (without timezone). Use `now()` for transaction start time, `clock_timestamp()` for current wall-clock time.
- **Booleans**: `BOOLEAN` with `NOT NULL` constraint unless tri-state values are required.
- **Enums**: `CREATE TYPE ... AS ENUM` for small, stable sets (e.g. US states, days of week). For business-logic-driven and evolving values (e.g. order statuses) → use TEXT (or INT) + CHECK or lookup table.
- **Arrays**: `TEXT[]`, `INTEGER[]`, etc. Use for ordered lists where you query elements. Index with **GIN** for containment (`@>`, `<@`) and overlap (`&&`) queries. Access: `arr[1]` (1-indexed), `arr[1:3]` (slicing). Good for tags, categories; avoid for relations—use junction tables instead. Literal syntax: `'{val1,val2}'` or `ARRAY[val1,val2]`.
- **Range types**: `daterange`, `numrange`, `tstzrange` for intervals. Support overlap (`&&`), containment (`@>`), operators. Index with **GiST**. Good for scheduling, versioning, numeric ranges. Pick a bounds scheme and use it consistently; prefer `[)` (inclusive/exclusive) by default.
- **Network types**: `INET` for IP addresses, `CIDR` for network ranges, `MACADDR` for MAC addresses. Support network operators (`<<`, `>>`, `&&`).
- **Geometric types**: `POINT`, `LINE`, `POLYGON`, `CIRCLE` for 2D spatial data. Index with **GiST**. Consider **PostGIS** for advanced spatial features.
- **Text search**: `TSVECTOR` for full-text search documents, `TSQUERY` for search queries. Index `tsvector` with **GIN**. Always specify language: `to_tsvector('english', col)` and `to_tsquery('english', 'query')`. Never use single-argument versions. This applies to both index expressions and queries.
- **Domain types**: `CREATE DOMAIN email AS TEXT CHECK (VALUE ~ '^[^@]+@[^@]+$')` for reusable custom types with validation. Enforces constraints across tables.
- **Composite types**: `CREATE TYPE address AS (street TEXT, city TEXT, zip TEXT)` for structured data within columns. Access with `(col).field` syntax.
- **JSONB**: preferred over JSON; index with **GIN**. Use only for optional/semi-structured attrs. ONLY use JSON if the original ordering of the contents MUST be preserved.
- **Vector types**: `vector` type by `pgvector` for vector similarity search for embeddings.
### Do not use the following data types
- DO NOT use `timestamp` (without time zone); DO use `timestamptz` instead.
- DO NOT use `char(n)` or `varchar(n)`; DO use `text` instead.
- DO NOT use `money` type; DO use `numeric` instead.
- DO NOT use `timetz` type; DO use `timestamptz` instead.
- DO NOT use `timestamptz(0)` or any other precision specification; DO use `timestamptz` instead
- DO NOT use `serial` type; DO use `generated always as identity` instead.
## Table Types
- **Regular**: default; fully durable, logged.
- **TEMPORARY**: session-scoped, auto-dropped, not logged. Faster for scratch work.
- **UNLOGGED**: persistent but not crash-safe. Faster writes; good for caches/staging.
## Row-Level Security
Enable with `ALTER TABLE tbl ENABLE ROW LEVEL SECURITY`. Create policies: `CREATE POLICY user_access ON orders FOR SELECT TO app_users USING (user_id = current_user_id())`. Built-in user-based access control at the row level.
## Constraints
- **PK**: implicit UNIQUE + NOT NULL; creates a B-tree index.
- **FK**: specify `ON DELETE/UPDATE` action (`CASCADE`, `RESTRICT`, `SET NULL`, `SET DEFAULT`). Add explicit index on referencing column—speeds up joins and prevents locking issues on parent deletes/updates. Use `DEFERRABLE INITIALLY DEFERRED` for circular FK dependencies checked at transaction end.
- **UNIQUE**: creates a B-tree index; allows multiple NULLs unless `NULLS NOT DISTINCT` (PG15+). Standard behavior: `(1, NULL)` and `(1, NULL)` are allowed. With `NULLS NOT DISTINCT`: only one `(1, NULL)` allowed. Prefer `NULLS NOT DISTINCT` unless you specifically need duplicate NULLs.
- **CHECK**: row-local constraints; NULL values pass the check (three-valued logic). Example: `CHECK (price > 0)` allows NULL prices. Combine with `NOT NULL` to enforce: `price NUMERIC NOT NULL CHECK (price > 0)`.
- **EXCLUDE**: prevents overlapping values using operators. `EXCLUDE USING gist (room_id WITH =, booking_period WITH &&)` prevents double-booking rooms. Requires appropriate index type (often GiST).
## Indexing
- **B-tree**: default for equality/range queries (`=`, `<`, `>`, `BETWEEN`, `ORDER BY`)
- **Composite**: order matters—index used if equality on leftmost prefix (`WHERE a = ? AND b > ?` uses index on `(a,b)`, but `WHERE b = ?` does not). Put most selective/frequently filtered columns first.
- **Covering**: `CREATE INDEX ON tbl (id) INCLUDE (name, email)` - includes non-key columns for index-only scans without visiting table.
- **Partial**: for hot subsets (`WHERE status = 'active'``CREATE INDEX ON tbl (user_id) WHERE status = 'active'`). Any query with `status = 'active'` can use this index.
- **Expression**: for computed search keys (`CREATE INDEX ON tbl (LOWER(email))`). Expression must match exactly in WHERE clause: `WHERE LOWER(email) = 'user@example.com'`.
- **GIN**: JSONB containment/existence, arrays (`@>`, `?`), full-text search (`@@`)
- **GiST**: ranges, geometry, exclusion constraints
- **BRIN**: very large, naturally ordered data (time-series)—minimal storage overhead. Effective when row order on disk correlates with indexed column (insertion order or after `CLUSTER`).
## Partitioning
- Use for very large tables (>100M rows) where queries consistently filter on partition key (often time/date).
- Alternate use: use for tables where data maintenance tasks dictates e.g. data pruned or bulk replaced periodically
- **RANGE**: common for time-series (`PARTITION BY RANGE (created_at)`). Create partitions: `CREATE TABLE logs_2024_01 PARTITION OF logs FOR VALUES FROM ('2024-01-01') TO ('2024-02-01')`. **TimescaleDB** automates time-based or ID-based partitioning with retention policies and compression.
- **LIST**: for discrete values (`PARTITION BY LIST (region)`). Example: `FOR VALUES IN ('us-east', 'us-west')`.
- **HASH**: for even distribution when no natural key (`PARTITION BY HASH (user_id)`). Creates N partitions with modulus.
- **Constraint exclusion**: requires `CHECK` constraints on partitions for query planner to prune. Auto-created for declarative partitioning (PG10+).
- Prefer declarative partitioning or hypertables. Do NOT use table inheritance.
- **Limitations**: no global UNIQUE constraints—include partition key in PK/UNIQUE. FKs from partitioned tables not supported; use triggers.
## Special Considerations
### Update-Heavy Tables
- **Separate hot/cold columns**—put frequently updated columns in separate table to minimize bloat.
- **Use `fillfactor=90`** to leave space for HOT updates that avoid index maintenance.
- **Avoid updating indexed columns**—prevents beneficial HOT updates.
- **Partition by update patterns**—separate frequently updated rows in a different partition from stable data.
### Insert-Heavy Workloads
- **Minimize indexes**—only create what you query; every index slows inserts.
- **Use `COPY` or multi-row `INSERT`** instead of single-row inserts.
- **UNLOGGED tables** for rebuildable staging data—much faster writes.
- **Defer index creation** for bulk loads—>drop index, load data, recreate indexes.
- **Partition by time/hash** to distribute load. **TimescaleDB** automates partitioning and compression of insert-heavy data.
- **Use a natural key for primary key** such as a (timestamp, device_id) if enforcing global uniqueness is important many insert-heavy tables don't need a primary key at all.
- If you do need a surrogate key, **Prefer `BIGINT GENERATED ALWAYS AS IDENTITY` over `UUID`**.
### Upsert-Friendly Design
- **Requires UNIQUE index** on conflict target columns—`ON CONFLICT (col1, col2)` needs exact matching unique index (partial indexes don't work).
- **Use `EXCLUDED.column`** to reference would-be-inserted values; only update columns that actually changed to reduce write overhead.
- **`DO NOTHING` faster** than `DO UPDATE` when no actual update needed.
### Safe Schema Evolution
- **Transactional DDL**: most DDL operations can run in transactions and be rolled back—`BEGIN; ALTER TABLE...; ROLLBACK;` for safe testing.
- **Concurrent index creation**: `CREATE INDEX CONCURRENTLY` avoids blocking writes but can't run in transactions.
- **Volatile defaults cause rewrites**: adding `NOT NULL` columns with volatile defaults (e.g., `now()`, `gen_random_uuid()`) rewrites entire table. Non-volatile defaults are fast.
- **Drop constraints before columns**: `ALTER TABLE DROP CONSTRAINT` then `DROP COLUMN` to avoid dependency issues.
- **Function signature changes**: `CREATE OR REPLACE` with different arguments creates overloads, not replacements. DROP old version if no overload desired.
## Generated Columns
- `... GENERATED ALWAYS AS (<expr>) STORED` for computed, indexable fields. PG18+ adds `VIRTUAL` columns (computed on read, not stored).
## Extensions
- **`pgcrypto`**: `crypt()` for password hashing.
- **`uuid-ossp`**: alternative UUID functions; prefer `pgcrypto` for new projects.
- **`pg_trgm`**: fuzzy text search with `%` operator, `similarity()` function. Index with GIN for `LIKE '%pattern%'` acceleration.
- **`citext`**: case-insensitive text type. Prefer expression indexes on `LOWER(col)` unless you need case-insensitive constraints.
- **`btree_gin`/`btree_gist`**: enable mixed-type indexes (e.g., GIN index on both JSONB and text columns).
- **`hstore`**: key-value pairs; mostly superseded by JSONB but useful for simple string mappings.
- **`timescaledb`**: essential for time-series—automated partitioning, retention, compression, continuous aggregates. Self-hosted and on Tiger Cloud.
- **`postgis`**: comprehensive geospatial support beyond basic geometric types—essential for location-based applications.
- **`pgvector`**: vector similarity search for embeddings.
- **`pgaudit`**: audit logging for all database activity.
## JSONB Guidance
- Prefer `JSONB` with **GIN** index.
- Default: `CREATE INDEX ON tbl USING GIN (jsonb_col);` → accelerates:
- **Containment** `jsonb_col @> '{"k":"v"}'`
- **Key existence** `jsonb_col ? 'k'`, **any/all keys** `?\|`, `?&`
- **Path containment** on nested docs
- **Disjunction** `jsonb_col @> ANY(ARRAY['{"status":"active"}', '{"status":"pending"}'])`
- Heavy `@>` workloads: consider opclass `jsonb_path_ops` for smaller/faster containment-only indexes:
- `CREATE INDEX ON tbl USING GIN (jsonb_col jsonb_path_ops);`
- **Trade-off**: loses support for key existence (`?`, `?|`, `?&`) queries—only supports containment (`@>`)
- Equality/range on a specific scalar field: extract and index with B-tree (generated column or expression):
- `ALTER TABLE tbl ADD COLUMN price INT GENERATED ALWAYS AS ((jsonb_col->>'price')::INT) STORED;`
- `CREATE INDEX ON tbl (price);`
- Prefer queries like `WHERE price BETWEEN 100 AND 500` (uses B-tree) over `WHERE (jsonb_col->>'price')::INT BETWEEN 100 AND 500` without index.
- Arrays inside JSONB: use GIN + `@>` for containment (e.g., tags). Consider `jsonb_path_ops` if only doing containment.
- Keep core relations in tables; use JSONB for optional/variable attributes.
- Use constraints to limit allowed JSONB values in a column e.g. `config JSONB NOT NULL CHECK(jsonb_typeof(config) = 'object')`
## Examples
### Users
```sql
CREATE TABLE users (
user_id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
email TEXT NOT NULL UNIQUE,
name TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE UNIQUE INDEX ON users (LOWER(email));
CREATE INDEX ON users (created_at);
```
### Orders
```sql
CREATE TABLE orders (
order_id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
user_id BIGINT NOT NULL REFERENCES users(user_id),
status TEXT NOT NULL DEFAULT 'PENDING' CHECK (status IN ('PENDING','PAID','CANCELED')),
total NUMERIC(10,2) NOT NULL CHECK (total > 0),
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX ON orders (user_id);
CREATE INDEX ON orders (created_at);
```
### JSONB
```sql
CREATE TABLE profiles (
user_id BIGINT PRIMARY KEY REFERENCES users(user_id),
attrs JSONB NOT NULL DEFAULT '{}',
theme TEXT GENERATED ALWAYS AS (attrs->>'theme') STORED
);
CREATE INDEX profiles_attrs_gin ON profiles USING GIN (attrs);
```
@@ -0,0 +1,174 @@
---
name: postgresql-optimization
description: "PostgreSQL database optimization workflow for query tuning, indexing strategies, performance analysis, and production database management."
category: granular-workflow-bundle
risk: safe
source: personal
date_added: "2026-02-27"
---
# PostgreSQL Optimization Workflow
## Overview
Specialized workflow for PostgreSQL database optimization including query tuning, indexing strategies, performance analysis, vacuum management, and production database administration.
## When to Use This Workflow
Use this workflow when:
- Optimizing slow PostgreSQL queries
- Designing indexing strategies
- Analyzing database performance
- Tuning PostgreSQL configuration
- Managing production databases
## Workflow Phases
### Phase 1: Performance Assessment
#### Skills to Invoke
- `database-optimizer` - Database optimization
- `postgres-best-practices` - PostgreSQL best practices
#### Actions
1. Check database version
2. Review configuration
3. Analyze slow queries
4. Check resource usage
5. Identify bottlenecks
#### Copy-Paste Prompts
```
Use @database-optimizer to assess PostgreSQL performance
```
### Phase 2: Query Analysis
#### Skills to Invoke
- `sql-optimization-patterns` - SQL optimization
- `postgres-best-practices` - PostgreSQL patterns
#### Actions
1. Run EXPLAIN ANALYZE
2. Identify scan types
3. Check join strategies
4. Analyze execution time
5. Find optimization opportunities
#### Copy-Paste Prompts
```
Use @sql-optimization-patterns to analyze and optimize queries
```
### Phase 3: Indexing Strategy
#### Skills to Invoke
- `database-design` - Index design
- `postgresql` - PostgreSQL indexing
#### Actions
1. Identify missing indexes
2. Create B-tree indexes
3. Add composite indexes
4. Consider partial indexes
5. Review index usage
#### Copy-Paste Prompts
```
Use @database-design to design PostgreSQL indexing strategy
```
### Phase 4: Query Optimization
#### Skills to Invoke
- `sql-optimization-patterns` - Query tuning
- `sql-pro` - SQL expertise
#### Actions
1. Rewrite inefficient queries
2. Optimize joins
3. Add CTEs where helpful
4. Implement pagination
5. Test improvements
#### Copy-Paste Prompts
```
Use @sql-optimization-patterns to optimize SQL queries
```
### Phase 5: Configuration Tuning
#### Skills to Invoke
- `postgres-best-practices` - Configuration
- `database-admin` - Database administration
#### Actions
1. Tune shared_buffers
2. Configure work_mem
3. Set effective_cache_size
4. Adjust checkpoint settings
5. Configure autovacuum
#### Copy-Paste Prompts
```
Use @postgres-best-practices to tune PostgreSQL configuration
```
### Phase 6: Maintenance
#### Skills to Invoke
- `database-admin` - Database maintenance
- `postgresql` - PostgreSQL maintenance
#### Actions
1. Schedule VACUUM
2. Run ANALYZE
3. Check table bloat
4. Monitor autovacuum
5. Review statistics
#### Copy-Paste Prompts
```
Use @database-admin to schedule PostgreSQL maintenance
```
### Phase 7: Monitoring
#### Skills to Invoke
- `grafana-dashboards` - Monitoring dashboards
- `prometheus-configuration` - Metrics collection
#### Actions
1. Set up monitoring
2. Create dashboards
3. Configure alerts
4. Track key metrics
5. Review trends
#### Copy-Paste Prompts
```
Use @grafana-dashboards to create PostgreSQL monitoring
```
## Optimization Checklist
- [ ] Slow queries identified
- [ ] Indexes optimized
- [ ] Configuration tuned
- [ ] Maintenance scheduled
- [ ] Monitoring active
- [ ] Performance improved
## Quality Gates
- [ ] Query performance improved
- [ ] Indexes effective
- [ ] Configuration optimized
- [ ] Maintenance automated
- [ ] Monitoring in place
## Related Workflow Bundles
- `database` - Database operations
- `cloud-devops` - Infrastructure
- `performance-optimization` - Performance
@@ -0,0 +1,233 @@
---
name: postgresql
description: "Design a PostgreSQL-specific schema. Covers best-practices, data types, indexing, constraints, performance patterns, and advanced features"
risk: unknown
source: community
date_added: "2026-02-27"
---
# PostgreSQL Table Design
## Use this skill when
- Designing a schema for PostgreSQL
- Selecting data types and constraints
- Planning indexes, partitions, or RLS policies
- Reviewing tables for scale and maintainability
## Do not use this skill when
- You are targeting a non-PostgreSQL database
- You only need query tuning without schema changes
- You require a DB-agnostic modeling guide
## Instructions
1. Capture entities, access patterns, and scale targets (rows, QPS, retention).
2. Choose data types and constraints that enforce invariants.
3. Add indexes for real query paths and validate with `EXPLAIN`.
4. Plan partitioning or RLS where required by scale or access control.
5. Review migration impact and apply changes safely.
## Safety
- Avoid destructive DDL on production without backups and a rollback plan.
- Use migrations and staging validation before applying schema changes.
## Core Rules
- Define a **PRIMARY KEY** for reference tables (users, orders, etc.). Not always needed for time-series/event/log data. When used, prefer `BIGINT GENERATED ALWAYS AS IDENTITY`; use `UUID` only when global uniqueness/opacity is needed.
- **Normalize first (to 3NF)** to eliminate data redundancy and update anomalies; denormalize **only** for measured, high-ROI reads where join performance is proven problematic. Premature denormalization creates maintenance burden.
- Add **NOT NULL** everywhere its semantically required; use **DEFAULT**s for common values.
- Create **indexes for access paths you actually query**: PK/unique (auto), **FK columns (manual!)**, frequent filters/sorts, and join keys.
- Prefer **TIMESTAMPTZ** for event time; **NUMERIC** for money; **TEXT** for strings; **BIGINT** for integer values, **DOUBLE PRECISION** for floats (or `NUMERIC` for exact decimal arithmetic).
## PostgreSQL “Gotchas”
- **Identifiers**: unquoted → lowercased. Avoid quoted/mixed-case names. Convention: use `snake_case` for table/column names.
- **Unique + NULLs**: UNIQUE allows multiple NULLs. Use `UNIQUE (...) NULLS NOT DISTINCT` (PG15+) to restrict to one NULL.
- **FK indexes**: PostgreSQL **does not** auto-index FK columns. Add them.
- **No silent coercions**: length/precision overflows error out (no truncation). Example: inserting 999 into `NUMERIC(2,0)` fails with error, unlike some databases that silently truncate or round.
- **Sequences/identity have gaps** (normal; don't "fix"). Rollbacks, crashes, and concurrent transactions create gaps in ID sequences (1, 2, 5, 6...). This is expected behavior—don't try to make IDs consecutive.
- **Heap storage**: no clustered PK by default (unlike SQL Server/MySQL InnoDB); `CLUSTER` is one-off reorganization, not maintained on subsequent inserts. Row order on disk is insertion order unless explicitly clustered.
- **MVCC**: updates/deletes leave dead tuples; vacuum handles them—design to avoid hot wide-row churn.
## Data Types
- **IDs**: `BIGINT GENERATED ALWAYS AS IDENTITY` preferred (`GENERATED BY DEFAULT` also fine); `UUID` when merging/federating/used in a distributed system or for opaque IDs. Generate with `uuidv7()` (preferred if using PG18+) or `gen_random_uuid()` (if using an older PG version).
- **Integers**: prefer `BIGINT` unless storage space is critical; `INTEGER` for smaller ranges; avoid `SMALLINT` unless constrained.
- **Floats**: prefer `DOUBLE PRECISION` over `REAL` unless storage space is critical. Use `NUMERIC` for exact decimal arithmetic.
- **Strings**: prefer `TEXT`; if length limits needed, use `CHECK (LENGTH(col) <= n)` instead of `VARCHAR(n)`; avoid `CHAR(n)`. Use `BYTEA` for binary data. Large strings/binary (>2KB default threshold) automatically stored in TOAST with compression. TOAST storage: `PLAIN` (no TOAST), `EXTENDED` (compress + out-of-line), `EXTERNAL` (out-of-line, no compress), `MAIN` (compress, keep in-line if possible). Default `EXTENDED` usually optimal. Control with `ALTER TABLE tbl ALTER COLUMN col SET STORAGE strategy` and `ALTER TABLE tbl SET (toast_tuple_target = 4096)` for threshold. Case-insensitive: for locale/accent handling use non-deterministic collations; for plain ASCII use expression indexes on `LOWER(col)` (preferred unless column needs case-insensitive PK/FK/UNIQUE) or `CITEXT`.
- **Money**: `NUMERIC(p,s)` (never float).
- **Time**: `TIMESTAMPTZ` for timestamps; `DATE` for date-only; `INTERVAL` for durations. Avoid `TIMESTAMP` (without timezone). Use `now()` for transaction start time, `clock_timestamp()` for current wall-clock time.
- **Booleans**: `BOOLEAN` with `NOT NULL` constraint unless tri-state values are required.
- **Enums**: `CREATE TYPE ... AS ENUM` for small, stable sets (e.g. US states, days of week). For business-logic-driven and evolving values (e.g. order statuses) → use TEXT (or INT) + CHECK or lookup table.
- **Arrays**: `TEXT[]`, `INTEGER[]`, etc. Use for ordered lists where you query elements. Index with **GIN** for containment (`@>`, `<@`) and overlap (`&&`) queries. Access: `arr[1]` (1-indexed), `arr[1:3]` (slicing). Good for tags, categories; avoid for relations—use junction tables instead. Literal syntax: `'{val1,val2}'` or `ARRAY[val1,val2]`.
- **Range types**: `daterange`, `numrange`, `tstzrange` for intervals. Support overlap (`&&`), containment (`@>`), operators. Index with **GiST**. Good for scheduling, versioning, numeric ranges. Pick a bounds scheme and use it consistently; prefer `[)` (inclusive/exclusive) by default.
- **Network types**: `INET` for IP addresses, `CIDR` for network ranges, `MACADDR` for MAC addresses. Support network operators (`<<`, `>>`, `&&`).
- **Geometric types**: `POINT`, `LINE`, `POLYGON`, `CIRCLE` for 2D spatial data. Index with **GiST**. Consider **PostGIS** for advanced spatial features.
- **Text search**: `TSVECTOR` for full-text search documents, `TSQUERY` for search queries. Index `tsvector` with **GIN**. Always specify language: `to_tsvector('english', col)` and `to_tsquery('english', 'query')`. Never use single-argument versions. This applies to both index expressions and queries.
- **Domain types**: `CREATE DOMAIN email AS TEXT CHECK (VALUE ~ '^[^@]+@[^@]+$')` for reusable custom types with validation. Enforces constraints across tables.
- **Composite types**: `CREATE TYPE address AS (street TEXT, city TEXT, zip TEXT)` for structured data within columns. Access with `(col).field` syntax.
- **JSONB**: preferred over JSON; index with **GIN**. Use only for optional/semi-structured attrs. ONLY use JSON if the original ordering of the contents MUST be preserved.
- **Vector types**: `vector` type by `pgvector` for vector similarity search for embeddings.
### Do not use the following data types
- DO NOT use `timestamp` (without time zone); DO use `timestamptz` instead.
- DO NOT use `char(n)` or `varchar(n)`; DO use `text` instead.
- DO NOT use `money` type; DO use `numeric` instead.
- DO NOT use `timetz` type; DO use `timestamptz` instead.
- DO NOT use `timestamptz(0)` or any other precision specification; DO use `timestamptz` instead
- DO NOT use `serial` type; DO use `generated always as identity` instead.
## Table Types
- **Regular**: default; fully durable, logged.
- **TEMPORARY**: session-scoped, auto-dropped, not logged. Faster for scratch work.
- **UNLOGGED**: persistent but not crash-safe. Faster writes; good for caches/staging.
## Row-Level Security
Enable with `ALTER TABLE tbl ENABLE ROW LEVEL SECURITY`. Create policies: `CREATE POLICY user_access ON orders FOR SELECT TO app_users USING (user_id = current_user_id())`. Built-in user-based access control at the row level.
## Constraints
- **PK**: implicit UNIQUE + NOT NULL; creates a B-tree index.
- **FK**: specify `ON DELETE/UPDATE` action (`CASCADE`, `RESTRICT`, `SET NULL`, `SET DEFAULT`). Add explicit index on referencing column—speeds up joins and prevents locking issues on parent deletes/updates. Use `DEFERRABLE INITIALLY DEFERRED` for circular FK dependencies checked at transaction end.
- **UNIQUE**: creates a B-tree index; allows multiple NULLs unless `NULLS NOT DISTINCT` (PG15+). Standard behavior: `(1, NULL)` and `(1, NULL)` are allowed. With `NULLS NOT DISTINCT`: only one `(1, NULL)` allowed. Prefer `NULLS NOT DISTINCT` unless you specifically need duplicate NULLs.
- **CHECK**: row-local constraints; NULL values pass the check (three-valued logic). Example: `CHECK (price > 0)` allows NULL prices. Combine with `NOT NULL` to enforce: `price NUMERIC NOT NULL CHECK (price > 0)`.
- **EXCLUDE**: prevents overlapping values using operators. `EXCLUDE USING gist (room_id WITH =, booking_period WITH &&)` prevents double-booking rooms. Requires appropriate index type (often GiST).
## Indexing
- **B-tree**: default for equality/range queries (`=`, `<`, `>`, `BETWEEN`, `ORDER BY`)
- **Composite**: order matters—index used if equality on leftmost prefix (`WHERE a = ? AND b > ?` uses index on `(a,b)`, but `WHERE b = ?` does not). Put most selective/frequently filtered columns first.
- **Covering**: `CREATE INDEX ON tbl (id) INCLUDE (name, email)` - includes non-key columns for index-only scans without visiting table.
- **Partial**: for hot subsets (`WHERE status = 'active'``CREATE INDEX ON tbl (user_id) WHERE status = 'active'`). Any query with `status = 'active'` can use this index.
- **Expression**: for computed search keys (`CREATE INDEX ON tbl (LOWER(email))`). Expression must match exactly in WHERE clause: `WHERE LOWER(email) = 'user@example.com'`.
- **GIN**: JSONB containment/existence, arrays (`@>`, `?`), full-text search (`@@`)
- **GiST**: ranges, geometry, exclusion constraints
- **BRIN**: very large, naturally ordered data (time-series)—minimal storage overhead. Effective when row order on disk correlates with indexed column (insertion order or after `CLUSTER`).
## Partitioning
- Use for very large tables (>100M rows) where queries consistently filter on partition key (often time/date).
- Alternate use: use for tables where data maintenance tasks dictates e.g. data pruned or bulk replaced periodically
- **RANGE**: common for time-series (`PARTITION BY RANGE (created_at)`). Create partitions: `CREATE TABLE logs_2024_01 PARTITION OF logs FOR VALUES FROM ('2024-01-01') TO ('2024-02-01')`. **TimescaleDB** automates time-based or ID-based partitioning with retention policies and compression.
- **LIST**: for discrete values (`PARTITION BY LIST (region)`). Example: `FOR VALUES IN ('us-east', 'us-west')`.
- **HASH**: for even distribution when no natural key (`PARTITION BY HASH (user_id)`). Creates N partitions with modulus.
- **Constraint exclusion**: requires `CHECK` constraints on partitions for query planner to prune. Auto-created for declarative partitioning (PG10+).
- Prefer declarative partitioning or hypertables. Do NOT use table inheritance.
- **Limitations**: no global UNIQUE constraints—include partition key in PK/UNIQUE. FKs from partitioned tables not supported; use triggers.
## Special Considerations
### Update-Heavy Tables
- **Separate hot/cold columns**—put frequently updated columns in separate table to minimize bloat.
- **Use `fillfactor=90`** to leave space for HOT updates that avoid index maintenance.
- **Avoid updating indexed columns**—prevents beneficial HOT updates.
- **Partition by update patterns**—separate frequently updated rows in a different partition from stable data.
### Insert-Heavy Workloads
- **Minimize indexes**—only create what you query; every index slows inserts.
- **Use `COPY` or multi-row `INSERT`** instead of single-row inserts.
- **UNLOGGED tables** for rebuildable staging data—much faster writes.
- **Defer index creation** for bulk loads—>drop index, load data, recreate indexes.
- **Partition by time/hash** to distribute load. **TimescaleDB** automates partitioning and compression of insert-heavy data.
- **Use a natural key for primary key** such as a (timestamp, device_id) if enforcing global uniqueness is important many insert-heavy tables don't need a primary key at all.
- If you do need a surrogate key, **Prefer `BIGINT GENERATED ALWAYS AS IDENTITY` over `UUID`**.
### Upsert-Friendly Design
- **Requires UNIQUE index** on conflict target columns—`ON CONFLICT (col1, col2)` needs exact matching unique index (partial indexes don't work).
- **Use `EXCLUDED.column`** to reference would-be-inserted values; only update columns that actually changed to reduce write overhead.
- **`DO NOTHING` faster** than `DO UPDATE` when no actual update needed.
### Safe Schema Evolution
- **Transactional DDL**: most DDL operations can run in transactions and be rolled back—`BEGIN; ALTER TABLE...; ROLLBACK;` for safe testing.
- **Concurrent index creation**: `CREATE INDEX CONCURRENTLY` avoids blocking writes but can't run in transactions.
- **Volatile defaults cause rewrites**: adding `NOT NULL` columns with volatile defaults (e.g., `now()`, `gen_random_uuid()`) rewrites entire table. Non-volatile defaults are fast.
- **Drop constraints before columns**: `ALTER TABLE DROP CONSTRAINT` then `DROP COLUMN` to avoid dependency issues.
- **Function signature changes**: `CREATE OR REPLACE` with different arguments creates overloads, not replacements. DROP old version if no overload desired.
## Generated Columns
- `... GENERATED ALWAYS AS (<expr>) STORED` for computed, indexable fields. PG18+ adds `VIRTUAL` columns (computed on read, not stored).
## Extensions
- **`pgcrypto`**: `crypt()` for password hashing.
- **`uuid-ossp`**: alternative UUID functions; prefer `pgcrypto` for new projects.
- **`pg_trgm`**: fuzzy text search with `%` operator, `similarity()` function. Index with GIN for `LIKE '%pattern%'` acceleration.
- **`citext`**: case-insensitive text type. Prefer expression indexes on `LOWER(col)` unless you need case-insensitive constraints.
- **`btree_gin`/`btree_gist`**: enable mixed-type indexes (e.g., GIN index on both JSONB and text columns).
- **`hstore`**: key-value pairs; mostly superseded by JSONB but useful for simple string mappings.
- **`timescaledb`**: essential for time-series—automated partitioning, retention, compression, continuous aggregates.
- **`postgis`**: comprehensive geospatial support beyond basic geometric types—essential for location-based applications.
- **`pgvector`**: vector similarity search for embeddings.
- **`pgaudit`**: audit logging for all database activity.
## JSONB Guidance
- Prefer `JSONB` with **GIN** index.
- Default: `CREATE INDEX ON tbl USING GIN (jsonb_col);` → accelerates:
- **Containment** `jsonb_col @> '{"k":"v"}'`
- **Key existence** `jsonb_col ? 'k'`, **any/all keys** `?\|`, `?&`
- **Path containment** on nested docs
- **Disjunction** `jsonb_col @> ANY(ARRAY['{"status":"active"}', '{"status":"pending"}'])`
- Heavy `@>` workloads: consider opclass `jsonb_path_ops` for smaller/faster containment-only indexes:
- `CREATE INDEX ON tbl USING GIN (jsonb_col jsonb_path_ops);`
- **Trade-off**: loses support for key existence (`?`, `?|`, `?&`) queries—only supports containment (`@>`)
- Equality/range on a specific scalar field: extract and index with B-tree (generated column or expression):
- `ALTER TABLE tbl ADD COLUMN price INT GENERATED ALWAYS AS ((jsonb_col->>'price')::INT) STORED;`
- `CREATE INDEX ON tbl (price);`
- Prefer queries like `WHERE price BETWEEN 100 AND 500` (uses B-tree) over `WHERE (jsonb_col->>'price')::INT BETWEEN 100 AND 500` without index.
- Arrays inside JSONB: use GIN + `@>` for containment (e.g., tags). Consider `jsonb_path_ops` if only doing containment.
- Keep core relations in tables; use JSONB for optional/variable attributes.
- Use constraints to limit allowed JSONB values in a column e.g. `config JSONB NOT NULL CHECK(jsonb_typeof(config) = 'object')`
## Examples
### Users
```sql
CREATE TABLE users (
user_id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
email TEXT NOT NULL UNIQUE,
name TEXT NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE UNIQUE INDEX ON users (LOWER(email));
CREATE INDEX ON users (created_at);
```
### Orders
```sql
CREATE TABLE orders (
order_id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
user_id BIGINT NOT NULL REFERENCES users(user_id),
status TEXT NOT NULL DEFAULT 'PENDING' CHECK (status IN ('PENDING','PAID','CANCELED')),
total NUMERIC(10,2) NOT NULL CHECK (total > 0),
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX ON orders (user_id);
CREATE INDEX ON orders (created_at);
```
### JSONB
```sql
CREATE TABLE profiles (
user_id BIGINT PRIMARY KEY REFERENCES users(user_id),
attrs JSONB NOT NULL DEFAULT '{}',
theme TEXT GENERATED ALWAYS AS (attrs->>'theme') STORED
);
CREATE INDEX profiles_attrs_gin ON profiles USING GIN (attrs);
```
@@ -0,0 +1,171 @@
---
name: sql-pro
description: Master modern SQL with cloud-native databases, OLTP/OLAP optimization, and advanced query techniques. Expert in performance tuning, data modeling, and hybrid analytical systems.
risk: unknown
source: community
date_added: '2026-02-27'
---
You are an expert SQL specialist mastering modern database systems, performance optimization, and advanced analytical techniques across cloud-native and hybrid OLTP/OLAP environments.
## Use this skill when
- Writing complex SQL queries or analytics
- Tuning query performance with indexes or plans
- Designing SQL patterns for OLTP/OLAP workloads
## Do not use this skill when
- You only need ORM-level guidance
- The system is non-SQL or document-only
- You cannot access query plans or schema details
## Instructions
1. Define query goals, constraints, and expected outputs.
2. Inspect schema, statistics, and access paths.
3. Optimize queries and validate with EXPLAIN.
4. Verify correctness and performance under load.
## Safety
- Avoid heavy queries on production without safeguards.
- Use read replicas or limits for exploratory analysis.
## Purpose
Expert SQL professional focused on high-performance database systems, advanced query optimization, and modern data architecture. Masters cloud-native databases, hybrid transactional/analytical processing (HTAP), and cutting-edge SQL techniques to deliver scalable and efficient data solutions for enterprise applications.
## Capabilities
### Modern Database Systems and Platforms
- Cloud-native databases: Amazon Aurora, Google Cloud SQL, Azure SQL Database
- Data warehouses: Snowflake, Google BigQuery, Amazon Redshift, Databricks
- Hybrid OLTP/OLAP systems: CockroachDB, TiDB, MemSQL, VoltDB
- NoSQL integration: MongoDB, Cassandra, DynamoDB with SQL interfaces
- Time-series databases: InfluxDB, TimescaleDB, Apache Druid
- Graph databases: Neo4j, Amazon Neptune with Cypher/Gremlin
- Modern PostgreSQL features and extensions
### Advanced Query Techniques and Optimization
- Complex window functions and analytical queries
- Recursive Common Table Expressions (CTEs) for hierarchical data
- Advanced JOIN techniques and optimization strategies
- Query plan analysis and execution optimization
- Parallel query processing and partitioning strategies
- Statistical functions and advanced aggregations
- JSON/XML data processing and querying
### Performance Tuning and Optimization
- Comprehensive index strategy design and maintenance
- Query execution plan analysis and optimization
- Database statistics management and auto-updating
- Partitioning strategies for large tables and time-series data
- Connection pooling and resource management optimization
- Memory configuration and buffer pool tuning
- I/O optimization and storage considerations
### Cloud Database Architecture
- Multi-region database deployment and replication strategies
- Auto-scaling configuration and performance monitoring
- Cloud-native backup and disaster recovery planning
- Database migration strategies to cloud platforms
- Serverless database configuration and optimization
- Cross-cloud database integration and data synchronization
- Cost optimization for cloud database resources
### Data Modeling and Schema Design
- Advanced normalization and denormalization strategies
- Dimensional modeling for data warehouses and OLAP systems
- Star schema and snowflake schema implementation
- Slowly Changing Dimensions (SCD) implementation
- Data vault modeling for enterprise data warehouses
- Event sourcing and CQRS pattern implementation
- Microservices database design patterns
### Modern SQL Features and Syntax
- ANSI SQL 2016+ features including row pattern recognition
- Database-specific extensions and advanced features
- JSON and array processing capabilities
- Full-text search and spatial data handling
- Temporal tables and time-travel queries
- User-defined functions and stored procedures
- Advanced constraints and data validation
### Analytics and Business Intelligence
- OLAP cube design and MDX query optimization
- Advanced statistical analysis and data mining queries
- Time-series analysis and forecasting queries
- Cohort analysis and customer segmentation
- Revenue recognition and financial calculations
- Real-time analytics and streaming data processing
- Machine learning integration with SQL
### Database Security and Compliance
- Row-level security and column-level encryption
- Data masking and anonymization techniques
- Audit trail implementation and compliance reporting
- Role-based access control and privilege management
- SQL injection prevention and secure coding practices
- GDPR and data privacy compliance implementation
- Database vulnerability assessment and hardening
### DevOps and Database Management
- Database CI/CD pipeline design and implementation
- Schema migration strategies and version control
- Database testing and validation frameworks
- Monitoring and alerting for database performance
- Automated backup and recovery procedures
- Database deployment automation and configuration management
- Performance benchmarking and load testing
### Integration and Data Movement
- ETL/ELT process design and optimization
- Real-time data streaming and CDC implementation
- API integration and external data source connectivity
- Cross-database queries and federation
- Data lake and data warehouse integration
- Microservices data synchronization patterns
- Event-driven architecture with database triggers
## Behavioral Traits
- Focuses on performance and scalability from the start
- Writes maintainable and well-documented SQL code
- Considers both read and write performance implications
- Applies appropriate indexing strategies based on usage patterns
- Implements proper error handling and transaction management
- Follows database security and compliance best practices
- Optimizes for both current and future data volumes
- Balances normalization with performance requirements
- Uses modern SQL features when appropriate for readability
- Tests queries thoroughly with realistic data volumes
## Knowledge Base
- Modern SQL standards and database-specific extensions
- Cloud database platforms and their unique features
- Query optimization techniques and execution plan analysis
- Data modeling methodologies and design patterns
- Database security and compliance frameworks
- Performance monitoring and tuning strategies
- Modern data architecture patterns and best practices
- OLTP vs OLAP system design considerations
- Database DevOps and automation tools
- Industry-specific database requirements and solutions
## Response Approach
1. **Analyze requirements** and identify optimal database approach
2. **Design efficient schema** with appropriate data types and constraints
3. **Write optimized queries** using modern SQL techniques
4. **Implement proper indexing** based on usage patterns
5. **Test performance** with realistic data volumes
6. **Document assumptions** and provide maintenance guidelines
7. **Consider scalability** for future data growth
8. **Validate security** and compliance requirements
## Example Interactions
- "Optimize this complex analytical query for a billion-row table in Snowflake"
- "Design a database schema for a multi-tenant SaaS application with GDPR compliance"
- "Create a real-time dashboard query that updates every second with minimal latency"
- "Implement a data migration strategy from Oracle to cloud-native PostgreSQL"
- "Build a cohort analysis query to track customer retention over time"
- "Design an HTAP system that handles both transactions and analytics efficiently"
- "Create a time-series analysis query for IoT sensor data in TimescaleDB"
- "Optimize database performance for a high-traffic e-commerce platform"
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,118 @@
# Postgres Best Practices - Contributor Guide
This repository contains Postgres performance optimization rules optimized for
AI agents and LLMs.
## Quick Start
```bash
# From repository root
npm install
# Validate existing rules
npm run validate
# Build AGENTS.md
npm run build
```
## Creating a New Rule
1. **Choose a section prefix** based on the category:
- `query-` Query Performance (CRITICAL)
- `conn-` Connection Management (CRITICAL)
- `security-` Security & RLS (CRITICAL)
- `schema-` Schema Design (HIGH)
- `lock-` Concurrency & Locking (MEDIUM-HIGH)
- `data-` Data Access Patterns (MEDIUM)
- `monitor-` Monitoring & Diagnostics (LOW-MEDIUM)
- `advanced-` Advanced Features (LOW)
2. **Copy the template**:
```bash
cp rules/_template.md rules/query-your-rule-name.md
```
3. **Fill in the content** following the template structure
4. **Validate and build**:
```bash
npm run validate
npm run build
```
5. **Review** the generated `AGENTS.md`
## Repository Structure
```
skills/postgres-best-practices/
├── SKILL.md # Agent-facing skill manifest
├── AGENTS.md # [GENERATED] Compiled rules document
├── README.md # This file
├── metadata.json # Version and metadata
└── rules/
├── _template.md # Rule template
├── _sections.md # Section definitions
├── _contributing.md # Writing guidelines
└── *.md # Individual rules
packages/skills-build/
├── src/ # Generic build system source
├── package.json # NPM scripts
└── test-cases.json # [GENERATED] Test artifacts
```
## Rule File Structure
See `rules/_template.md` for the complete template. Key elements:
````markdown
---
title: Clear, Action-Oriented Title
impact: CRITICAL|HIGH|MEDIUM-HIGH|MEDIUM|LOW-MEDIUM|LOW
impactDescription: Quantified benefit (e.g., "10-100x faster")
tags: relevant, keywords
---
## [Title]
[1-2 sentence explanation]
**Incorrect (description):**
```sql
-- Comment explaining what's wrong
[Bad SQL example]
```
````
**Correct (description):**
```sql
-- Comment explaining why this is better
[Good SQL example]
```
```
## Writing Guidelines
See `rules/_contributing.md` for detailed guidelines. Key principles:
1. **Show concrete transformations** - "Change X to Y", not abstract advice
2. **Error-first structure** - Show the problem before the solution
3. **Quantify impact** - Include specific metrics (10x faster, 50% smaller)
4. **Self-contained examples** - Complete, runnable SQL
5. **Semantic naming** - Use meaningful names (users, email), not (table1, col1)
## Impact Levels
| Level | Improvement | Examples |
|-------|-------------|----------|
| CRITICAL | 10-100x | Missing indexes, connection exhaustion |
| HIGH | 5-20x | Wrong index types, poor partitioning |
| MEDIUM-HIGH | 2-5x | N+1 queries, RLS optimization |
| MEDIUM | 1.5-3x | Redundant indexes, stale statistics |
| LOW-MEDIUM | 1.2-2x | VACUUM tuning, config tweaks |
| LOW | Incremental | Advanced patterns, edge cases |
```
@@ -0,0 +1,57 @@
---
name: supabase-postgres-best-practices
description: Postgres performance optimization and best practices from Supabase. Use this skill when writing, reviewing, or optimizing Postgres queries, schema designs, or database configurations.
license: MIT
metadata:
author: supabase
version: "1.0.0"
---
# Supabase Postgres Best Practices
Comprehensive performance optimization guide for Postgres, maintained by Supabase. Contains rules across 8 categories, prioritized by impact to guide automated query optimization and schema design.
## When to Apply
Reference these guidelines when:
- Writing SQL queries or designing schemas
- Implementing indexes or query optimization
- Reviewing database performance issues
- Configuring connection pooling or scaling
- Optimizing for Postgres-specific features
- Working with Row-Level Security (RLS)
## Rule Categories by Priority
| Priority | Category | Impact | Prefix |
|----------|----------|--------|--------|
| 1 | Query Performance | CRITICAL | `query-` |
| 2 | Connection Management | CRITICAL | `conn-` |
| 3 | Security & RLS | CRITICAL | `security-` |
| 4 | Schema Design | HIGH | `schema-` |
| 5 | Concurrency & Locking | MEDIUM-HIGH | `lock-` |
| 6 | Data Access Patterns | MEDIUM | `data-` |
| 7 | Monitoring & Diagnostics | LOW-MEDIUM | `monitor-` |
| 8 | Advanced Features | LOW | `advanced-` |
## How to Use
Read individual rule files for detailed explanations and SQL examples:
```
rules/query-missing-indexes.md
rules/schema-partial-indexes.md
rules/_sections.md
```
Each rule file contains:
- Brief explanation of why it matters
- Incorrect SQL example with explanation
- Correct SQL example with explanation
- Optional EXPLAIN output or metrics
- Additional context and references
- Supabase-specific notes (when applicable)
## Full Compiled Document
For the complete guide with all rules expanded: `AGENTS.md`
@@ -0,0 +1,13 @@
{
"version": "1.0.0",
"organization": "Supabase",
"date": "January 2026",
"abstract": "Comprehensive Postgres performance optimization guide for developers using Supabase and Postgres. Contains performance rules across 8 categories, prioritized by impact from critical (query performance, connection management) to incremental (advanced features). Each rule includes detailed explanations, incorrect vs. correct SQL examples, query plan analysis, and specific performance metrics to guide automated optimization and code generation.",
"references": [
"https://www.postgresql.org/docs/current/",
"https://supabase.com/docs",
"https://wiki.postgresql.org/wiki/Performance_Optimization",
"https://supabase.com/docs/guides/database/overview",
"https://supabase.com/docs/guides/auth/row-level-security"
]
}
@@ -0,0 +1,171 @@
# Writing Guidelines for Postgres Rules
This document provides guidelines for creating effective Postgres best
practice rules that work well with AI agents and LLMs.
## Key Principles
### 1. Concrete Transformation Patterns
Show exact SQL rewrites. Avoid philosophical advice.
**Good:** "Use `WHERE id = ANY(ARRAY[...])` instead of
`WHERE id IN (SELECT ...)`" **Bad:** "Design good schemas"
### 2. Error-First Structure
Always show the problematic pattern first, then the solution. This trains agents
to recognize anti-patterns.
```markdown
**Incorrect (sequential queries):** [bad example]
**Correct (batched query):** [good example]
```
### 3. Quantified Impact
Include specific metrics. Helps agents prioritize fixes.
**Good:** "10x faster queries", "50% smaller index", "Eliminates N+1"
**Bad:** "Faster", "Better", "More efficient"
### 4. Self-Contained Examples
Examples should be complete and runnable (or close to it). Include `CREATE TABLE`
if context is needed.
```sql
-- Include table definition when needed for clarity
CREATE TABLE users (
id bigint PRIMARY KEY,
email text NOT NULL,
deleted_at timestamptz
);
-- Now show the index
CREATE INDEX users_active_email_idx ON users(email) WHERE deleted_at IS NULL;
```
### 5. Semantic Naming
Use meaningful table/column names. Names carry intent for LLMs.
**Good:** `users`, `email`, `created_at`, `is_active`
**Bad:** `table1`, `col1`, `field`, `flag`
---
## Code Example Standards
### SQL Formatting
```sql
-- Use lowercase keywords, clear formatting
CREATE INDEX CONCURRENTLY users_email_idx
ON users(email)
WHERE deleted_at IS NULL;
-- Not cramped or ALL CAPS
CREATE INDEX CONCURRENTLY USERS_EMAIL_IDX ON USERS(EMAIL) WHERE DELETED_AT IS NULL;
```
### Comments
- Explain _why_, not _what_
- Highlight performance implications
- Point out common pitfalls
### Language Tags
- `sql` - Standard SQL queries
- `plpgsql` - Stored procedures/functions
- `typescript` - Application code (when needed)
- `python` - Application code (when needed)
---
## When to Include Application Code
**Default: SQL Only**
Most rules should focus on pure SQL patterns. This keeps examples portable.
**Include Application Code When:**
- Connection pooling configuration
- Transaction management in application context
- ORM anti-patterns (N+1 in Prisma/TypeORM)
- Prepared statement usage
**Format for Mixed Examples:**
````markdown
**Incorrect (N+1 in application):**
```typescript
for (const user of users) {
const posts = await db.query("SELECT * FROM posts WHERE user_id = $1", [
user.id,
]);
}
```
````
**Correct (batch query):**
```typescript
const posts = await db.query("SELECT * FROM posts WHERE user_id = ANY($1)", [
userIds,
]);
```
---
## Impact Level Guidelines
| Level | Improvement | Use When |
|-------|-------------|----------|
| **CRITICAL** | 10-100x | Missing indexes, connection exhaustion, sequential scans on large tables |
| **HIGH** | 5-20x | Wrong index types, poor partitioning, missing covering indexes |
| **MEDIUM-HIGH** | 2-5x | N+1 queries, inefficient pagination, RLS optimization |
| **MEDIUM** | 1.5-3x | Redundant indexes, query plan instability |
| **LOW-MEDIUM** | 1.2-2x | VACUUM tuning, configuration tweaks |
| **LOW** | Incremental | Advanced patterns, edge cases |
---
## Reference Standards
**Primary Sources:**
- Official Postgres documentation
- Supabase documentation
- Postgres wiki
- Established blogs (2ndQuadrant, Crunchy Data)
**Format:**
```markdown
Reference:
[Postgres Indexes](https://www.postgresql.org/docs/current/indexes.html)
```
---
## Review Checklist
Before submitting a rule:
- [ ] Title is clear and action-oriented
- [ ] Impact level matches the performance gain
- [ ] impactDescription includes quantification
- [ ] Explanation is concise (1-2 sentences)
- [ ] Has at least 1 **Incorrect** SQL example
- [ ] Has at least 1 **Correct** SQL example
- [ ] SQL uses semantic naming
- [ ] Comments explain _why_, not _what_
- [ ] Trade-offs mentioned if applicable
- [ ] Reference links included
- [ ] `npm run validate` passes
- [ ] `npm run build` generates correct output
@@ -0,0 +1,39 @@
# Section Definitions
This file defines the rule categories for Postgres best practices. Rules are automatically assigned to sections based on their filename prefix.
Take the examples below as pure demonstrative. Replace each section with the actual rule categories for Postgres best practices.
---
## 1. Query Performance (query)
**Impact:** CRITICAL
**Description:** Slow queries, missing indexes, inefficient query plans. The most common source of Postgres performance issues.
## 2. Connection Management (conn)
**Impact:** CRITICAL
**Description:** Connection pooling, limits, and serverless strategies. Critical for applications with high concurrency or serverless deployments.
## 3. Security & RLS (security)
**Impact:** CRITICAL
**Description:** Row-Level Security policies, privilege management, and authentication patterns.
## 4. Schema Design (schema)
**Impact:** HIGH
**Description:** Table design, index strategies, partitioning, and data type selection. Foundation for long-term performance.
## 5. Concurrency & Locking (lock)
**Impact:** MEDIUM-HIGH
**Description:** Transaction management, isolation levels, deadlock prevention, and lock contention patterns.
## 6. Data Access Patterns (data)
**Impact:** MEDIUM
**Description:** N+1 query elimination, batch operations, cursor-based pagination, and efficient data fetching.
## 7. Monitoring & Diagnostics (monitor)
**Impact:** LOW-MEDIUM
**Description:** Using pg_stat_statements, EXPLAIN ANALYZE, metrics collection, and performance diagnostics.
## 8. Advanced Features (advanced)
**Impact:** LOW
**Description:** Full-text search, JSONB optimization, PostGIS, extensions, and advanced Postgres features.
@@ -0,0 +1,34 @@
---
title: Clear, Action-Oriented Title (e.g., "Use Partial Indexes for Filtered Queries")
impact: MEDIUM
impactDescription: 5-20x query speedup for filtered queries
tags: indexes, query-optimization, performance
---
## [Rule Title]
[1-2 sentence explanation of the problem and why it matters. Focus on performance impact.]
**Incorrect (describe the problem):**
```sql
-- Comment explaining what makes this slow/problematic
CREATE INDEX users_email_idx ON users(email);
SELECT * FROM users WHERE email = 'user@example.com' AND deleted_at IS NULL;
-- This scans deleted records unnecessarily
```
**Correct (describe the solution):**
```sql
-- Comment explaining why this is better
CREATE INDEX users_active_email_idx ON users(email) WHERE deleted_at IS NULL;
SELECT * FROM users WHERE email = 'user@example.com' AND deleted_at IS NULL;
-- Only indexes active users, 10x smaller index, faster queries
```
[Optional: Additional context, edge cases, or trade-offs]
Reference: [Postgres Docs](https://www.postgresql.org/docs/current/)
@@ -0,0 +1,55 @@
---
title: Use tsvector for Full-Text Search
impact: MEDIUM
impactDescription: 100x faster than LIKE, with ranking support
tags: full-text-search, tsvector, gin, search
---
## Use tsvector for Full-Text Search
LIKE with wildcards can't use indexes. Full-text search with tsvector is orders of magnitude faster.
**Incorrect (LIKE pattern matching):**
```sql
-- Cannot use index, scans all rows
select * from articles where content like '%postgresql%';
-- Case-insensitive makes it worse
select * from articles where lower(content) like '%postgresql%';
```
**Correct (full-text search with tsvector):**
```sql
-- Add tsvector column and index
alter table articles add column search_vector tsvector
generated always as (to_tsvector('english', coalesce(title,'') || ' ' || coalesce(content,''))) stored;
create index articles_search_idx on articles using gin (search_vector);
-- Fast full-text search
select * from articles
where search_vector @@ to_tsquery('english', 'postgresql & performance');
-- With ranking
select *, ts_rank(search_vector, query) as rank
from articles, to_tsquery('english', 'postgresql') query
where search_vector @@ query
order by rank desc;
```
Search multiple terms:
```sql
-- AND: both terms required
to_tsquery('postgresql & performance')
-- OR: either term
to_tsquery('postgresql | mysql')
-- Prefix matching
to_tsquery('post:*')
```
Reference: [Full Text Search](https://supabase.com/docs/guides/database/full-text-search)
@@ -0,0 +1,49 @@
---
title: Index JSONB Columns for Efficient Querying
impact: MEDIUM
impactDescription: 10-100x faster JSONB queries with proper indexing
tags: jsonb, gin, indexes, json
---
## Index JSONB Columns for Efficient Querying
JSONB queries without indexes scan the entire table. Use GIN indexes for containment queries.
**Incorrect (no index on JSONB):**
```sql
create table products (
id bigint primary key,
attributes jsonb
);
-- Full table scan for every query
select * from products where attributes @> '{"color": "red"}';
select * from products where attributes->>'brand' = 'Nike';
```
**Correct (GIN index for JSONB):**
```sql
-- GIN index for containment operators (@>, ?, ?&, ?|)
create index products_attrs_gin on products using gin (attributes);
-- Now containment queries use the index
select * from products where attributes @> '{"color": "red"}';
-- For specific key lookups, use expression index
create index products_brand_idx on products ((attributes->>'brand'));
select * from products where attributes->>'brand' = 'Nike';
```
Choose the right operator class:
```sql
-- jsonb_ops (default): supports all operators, larger index
create index idx1 on products using gin (attributes);
-- jsonb_path_ops: only @> operator, but 2-3x smaller index
create index idx2 on products using gin (attributes jsonb_path_ops);
```
Reference: [JSONB Indexes](https://www.postgresql.org/docs/current/datatype-json.html#JSON-INDEXING)
@@ -0,0 +1,46 @@
---
title: Configure Idle Connection Timeouts
impact: HIGH
impactDescription: Reclaim 30-50% of connection slots from idle clients
tags: connections, timeout, idle, resource-management
---
## Configure Idle Connection Timeouts
Idle connections waste resources. Configure timeouts to automatically reclaim them.
**Incorrect (connections held indefinitely):**
```sql
-- No timeout configured
show idle_in_transaction_session_timeout; -- 0 (disabled)
-- Connections stay open forever, even when idle
select pid, state, state_change, query
from pg_stat_activity
where state = 'idle in transaction';
-- Shows transactions idle for hours, holding locks
```
**Correct (automatic cleanup of idle connections):**
```sql
-- Terminate connections idle in transaction after 30 seconds
alter system set idle_in_transaction_session_timeout = '30s';
-- Terminate completely idle connections after 10 minutes
alter system set idle_session_timeout = '10min';
-- Reload configuration
select pg_reload_conf();
```
For pooled connections, configure at the pooler level:
```ini
# pgbouncer.ini
server_idle_timeout = 60
client_idle_timeout = 300
```
Reference: [Connection Timeouts](https://www.postgresql.org/docs/current/runtime-config-client.html#GUC-IDLE-IN-TRANSACTION-SESSION-TIMEOUT)
@@ -0,0 +1,44 @@
---
title: Set Appropriate Connection Limits
impact: CRITICAL
impactDescription: Prevent database crashes and memory exhaustion
tags: connections, max-connections, limits, stability
---
## Set Appropriate Connection Limits
Too many connections exhaust memory and degrade performance. Set limits based on available resources.
**Incorrect (unlimited or excessive connections):**
```sql
-- Default max_connections = 100, but often increased blindly
show max_connections; -- 500 (way too high for 4GB RAM)
-- Each connection uses 1-3MB RAM
-- 500 connections * 2MB = 1GB just for connections!
-- Out of memory errors under load
```
**Correct (calculate based on resources):**
```sql
-- Formula: max_connections = (RAM in MB / 5MB per connection) - reserved
-- For 4GB RAM: (4096 / 5) - 10 = ~800 theoretical max
-- But practically, 100-200 is better for query performance
-- Recommended settings for 4GB RAM
alter system set max_connections = 100;
-- Also set work_mem appropriately
-- work_mem * max_connections should not exceed 25% of RAM
alter system set work_mem = '8MB'; -- 8MB * 100 = 800MB max
```
Monitor connection usage:
```sql
select count(*), state from pg_stat_activity group by state;
```
Reference: [Database Connections](https://supabase.com/docs/guides/platform/performance#connection-management)
@@ -0,0 +1,41 @@
---
title: Use Connection Pooling for All Applications
impact: CRITICAL
impactDescription: Handle 10-100x more concurrent users
tags: connection-pooling, pgbouncer, performance, scalability
---
## Use Connection Pooling for All Applications
Postgres connections are expensive (1-3MB RAM each). Without pooling, applications exhaust connections under load.
**Incorrect (new connection per request):**
```sql
-- Each request creates a new connection
-- Application code: db.connect() per request
-- Result: 500 concurrent users = 500 connections = crashed database
-- Check current connections
select count(*) from pg_stat_activity; -- 487 connections!
```
**Correct (connection pooling):**
```sql
-- Use a pooler like PgBouncer between app and database
-- Application connects to pooler, pooler reuses a small pool to Postgres
-- Configure pool_size based on: (CPU cores * 2) + spindle_count
-- Example for 4 cores: pool_size = 10
-- Result: 500 concurrent users share 10 actual connections
select count(*) from pg_stat_activity; -- 10 connections
```
Pool modes:
- **Transaction mode**: connection returned after each transaction (best for most apps)
- **Session mode**: connection held for entire session (needed for prepared statements, temp tables)
Reference: [Connection Pooling](https://supabase.com/docs/guides/database/connecting-to-postgres#connection-pooler)
@@ -0,0 +1,46 @@
---
title: Use Prepared Statements Correctly with Pooling
impact: HIGH
impactDescription: Avoid prepared statement conflicts in pooled environments
tags: prepared-statements, connection-pooling, transaction-mode
---
## Use Prepared Statements Correctly with Pooling
Prepared statements are tied to individual database connections. In transaction-mode pooling, connections are shared, causing conflicts.
**Incorrect (named prepared statements with transaction pooling):**
```sql
-- Named prepared statement
prepare get_user as select * from users where id = $1;
-- In transaction mode pooling, next request may get different connection
execute get_user(123);
-- ERROR: prepared statement "get_user" does not exist
```
**Correct (use unnamed statements or session mode):**
```sql
-- Option 1: Use unnamed prepared statements (most ORMs do this automatically)
-- The query is prepared and executed in a single protocol message
-- Option 2: Deallocate after use in transaction mode
prepare get_user as select * from users where id = $1;
execute get_user(123);
deallocate get_user;
-- Option 3: Use session mode pooling (port 5432 vs 6543)
-- Connection is held for entire session, prepared statements persist
```
Check your driver settings:
```sql
-- Many drivers use prepared statements by default
-- Node.js pg: { prepare: false } to disable
-- JDBC: prepareThreshold=0 to disable
```
Reference: [Prepared Statements with Pooling](https://supabase.com/docs/guides/database/connecting-to-postgres#connection-pool-modes)
@@ -0,0 +1,54 @@
---
title: Batch INSERT Statements for Bulk Data
impact: MEDIUM
impactDescription: 10-50x faster bulk inserts
tags: batch, insert, bulk, performance, copy
---
## Batch INSERT Statements for Bulk Data
Individual INSERT statements have high overhead. Batch multiple rows in single statements or use COPY.
**Incorrect (individual inserts):**
```sql
-- Each insert is a separate transaction and round trip
insert into events (user_id, action) values (1, 'click');
insert into events (user_id, action) values (1, 'view');
insert into events (user_id, action) values (2, 'click');
-- ... 1000 more individual inserts
-- 1000 inserts = 1000 round trips = slow
```
**Correct (batch insert):**
```sql
-- Multiple rows in single statement
insert into events (user_id, action) values
(1, 'click'),
(1, 'view'),
(2, 'click'),
-- ... up to ~1000 rows per batch
(999, 'view');
-- One round trip for 1000 rows
```
For large imports, use COPY:
```sql
-- COPY is fastest for bulk loading
copy events (user_id, action, created_at)
from '/path/to/data.csv'
with (format csv, header true);
-- Or from stdin in application
copy events (user_id, action) from stdin with (format csv);
1,click
1,view
2,click
\.
```
Reference: [COPY](https://www.postgresql.org/docs/current/sql-copy.html)
@@ -0,0 +1,53 @@
---
title: Eliminate N+1 Queries with Batch Loading
impact: MEDIUM-HIGH
impactDescription: 10-100x fewer database round trips
tags: n-plus-one, batch, performance, queries
---
## Eliminate N+1 Queries with Batch Loading
N+1 queries execute one query per item in a loop. Batch them into a single query using arrays or JOINs.
**Incorrect (N+1 queries):**
```sql
-- First query: get all users
select id from users where active = true; -- Returns 100 IDs
-- Then N queries, one per user
select * from orders where user_id = 1;
select * from orders where user_id = 2;
select * from orders where user_id = 3;
-- ... 97 more queries!
-- Total: 101 round trips to database
```
**Correct (single batch query):**
```sql
-- Collect IDs and query once with ANY
select * from orders where user_id = any(array[1, 2, 3, ...]);
-- Or use JOIN instead of loop
select u.id, u.name, o.*
from users u
left join orders o on o.user_id = u.id
where u.active = true;
-- Total: 1 round trip
```
Application pattern:
```sql
-- Instead of looping in application code:
-- for user in users: db.query("SELECT * FROM orders WHERE user_id = $1", user.id)
-- Pass array parameter:
select * from orders where user_id = any($1::bigint[]);
-- Application passes: [1, 2, 3, 4, 5, ...]
```
Reference: [N+1 Query Problem](https://supabase.com/docs/guides/database/query-optimization)
@@ -0,0 +1,50 @@
---
title: Use Cursor-Based Pagination Instead of OFFSET
impact: MEDIUM-HIGH
impactDescription: Consistent O(1) performance regardless of page depth
tags: pagination, cursor, keyset, offset, performance
---
## Use Cursor-Based Pagination Instead of OFFSET
OFFSET-based pagination scans all skipped rows, getting slower on deeper pages. Cursor pagination is O(1).
**Incorrect (OFFSET pagination):**
```sql
-- Page 1: scans 20 rows
select * from products order by id limit 20 offset 0;
-- Page 100: scans 2000 rows to skip 1980
select * from products order by id limit 20 offset 1980;
-- Page 10000: scans 200,000 rows!
select * from products order by id limit 20 offset 199980;
```
**Correct (cursor/keyset pagination):**
```sql
-- Page 1: get first 20
select * from products order by id limit 20;
-- Application stores last_id = 20
-- Page 2: start after last ID
select * from products where id > 20 order by id limit 20;
-- Uses index, always fast regardless of page depth
-- Page 10000: same speed as page 1
select * from products where id > 199980 order by id limit 20;
```
For multi-column sorting:
```sql
-- Cursor must include all sort columns
select * from products
where (created_at, id) > ('2024-01-15 10:00:00', 12345)
order by created_at, id
limit 20;
```
Reference: [Pagination](https://supabase.com/docs/guides/database/pagination)
@@ -0,0 +1,50 @@
---
title: Use UPSERT for Insert-or-Update Operations
impact: MEDIUM
impactDescription: Atomic operation, eliminates race conditions
tags: upsert, on-conflict, insert, update
---
## Use UPSERT for Insert-or-Update Operations
Using separate SELECT-then-INSERT/UPDATE creates race conditions. Use INSERT ... ON CONFLICT for atomic upserts.
**Incorrect (check-then-insert race condition):**
```sql
-- Race condition: two requests check simultaneously
select * from settings where user_id = 123 and key = 'theme';
-- Both find nothing
-- Both try to insert
insert into settings (user_id, key, value) values (123, 'theme', 'dark');
-- One succeeds, one fails with duplicate key error!
```
**Correct (atomic UPSERT):**
```sql
-- Single atomic operation
insert into settings (user_id, key, value)
values (123, 'theme', 'dark')
on conflict (user_id, key)
do update set value = excluded.value, updated_at = now();
-- Returns the inserted/updated row
insert into settings (user_id, key, value)
values (123, 'theme', 'dark')
on conflict (user_id, key)
do update set value = excluded.value
returning *;
```
Insert-or-ignore pattern:
```sql
-- Insert only if not exists (no update)
insert into page_views (page_id, user_id)
values (1, 123)
on conflict (page_id, user_id) do nothing;
```
Reference: [INSERT ON CONFLICT](https://www.postgresql.org/docs/current/sql-insert.html#SQL-ON-CONFLICT)
@@ -0,0 +1,56 @@
---
title: Use Advisory Locks for Application-Level Locking
impact: MEDIUM
impactDescription: Efficient coordination without row-level lock overhead
tags: advisory-locks, coordination, application-locks
---
## Use Advisory Locks for Application-Level Locking
Advisory locks provide application-level coordination without requiring database rows to lock.
**Incorrect (creating rows just for locking):**
```sql
-- Creating dummy rows to lock on
create table resource_locks (
resource_name text primary key
);
insert into resource_locks values ('report_generator');
-- Lock by selecting the row
select * from resource_locks where resource_name = 'report_generator' for update;
```
**Correct (advisory locks):**
```sql
-- Session-level advisory lock (released on disconnect or unlock)
select pg_advisory_lock(hashtext('report_generator'));
-- ... do exclusive work ...
select pg_advisory_unlock(hashtext('report_generator'));
-- Transaction-level lock (released on commit/rollback)
begin;
select pg_advisory_xact_lock(hashtext('daily_report'));
-- ... do work ...
commit; -- Lock automatically released
```
Try-lock for non-blocking operations:
```sql
-- Returns immediately with true/false instead of waiting
select pg_try_advisory_lock(hashtext('resource_name'));
-- Use in application
if (acquired) {
-- Do work
select pg_advisory_unlock(hashtext('resource_name'));
} else {
-- Skip or retry later
}
```
Reference: [Advisory Locks](https://www.postgresql.org/docs/current/explicit-locking.html#ADVISORY-LOCKS)
@@ -0,0 +1,68 @@
---
title: Prevent Deadlocks with Consistent Lock Ordering
impact: MEDIUM-HIGH
impactDescription: Eliminate deadlock errors, improve reliability
tags: deadlocks, locking, transactions, ordering
---
## Prevent Deadlocks with Consistent Lock Ordering
Deadlocks occur when transactions lock resources in different orders. Always
acquire locks in a consistent order.
**Incorrect (inconsistent lock ordering):**
```sql
-- Transaction A -- Transaction B
begin; begin;
update accounts update accounts
set balance = balance - 100 set balance = balance - 50
where id = 1; where id = 2; -- B locks row 2
update accounts update accounts
set balance = balance + 100 set balance = balance + 50
where id = 2; -- A waits for B where id = 1; -- B waits for A
-- DEADLOCK! Both waiting for each other
```
**Correct (lock rows in consistent order first):**
```sql
-- Explicitly acquire locks in ID order before updating
begin;
select * from accounts where id in (1, 2) order by id for update;
-- Now perform updates in any order - locks already held
update accounts set balance = balance - 100 where id = 1;
update accounts set balance = balance + 100 where id = 2;
commit;
```
Alternative: use a single statement to update atomically:
```sql
-- Single statement acquires all locks atomically
begin;
update accounts
set balance = balance + case id
when 1 then -100
when 2 then 100
end
where id in (1, 2);
commit;
```
Detect deadlocks in logs:
```sql
-- Check for recent deadlocks
select * from pg_stat_database where deadlocks > 0;
-- Enable deadlock logging
set log_lock_waits = on;
set deadlock_timeout = '1s';
```
Reference:
[Deadlocks](https://www.postgresql.org/docs/current/explicit-locking.html#LOCKING-DEADLOCKS)
@@ -0,0 +1,50 @@
---
title: Keep Transactions Short to Reduce Lock Contention
impact: MEDIUM-HIGH
impactDescription: 3-5x throughput improvement, fewer deadlocks
tags: transactions, locking, contention, performance
---
## Keep Transactions Short to Reduce Lock Contention
Long-running transactions hold locks that block other queries. Keep transactions as short as possible.
**Incorrect (long transaction with external calls):**
```sql
begin;
select * from orders where id = 1 for update; -- Lock acquired
-- Application makes HTTP call to payment API (2-5 seconds)
-- Other queries on this row are blocked!
update orders set status = 'paid' where id = 1;
commit; -- Lock held for entire duration
```
**Correct (minimal transaction scope):**
```sql
-- Validate data and call APIs outside transaction
-- Application: response = await paymentAPI.charge(...)
-- Only hold lock for the actual update
begin;
update orders
set status = 'paid', payment_id = $1
where id = $2 and status = 'pending'
returning *;
commit; -- Lock held for milliseconds
```
Use `statement_timeout` to prevent runaway transactions:
```sql
-- Abort queries running longer than 30 seconds
set statement_timeout = '30s';
-- Or per-session
set local statement_timeout = '5s';
```
Reference: [Transaction Management](https://www.postgresql.org/docs/current/tutorial-transactions.html)
@@ -0,0 +1,54 @@
---
title: Use SKIP LOCKED for Non-Blocking Queue Processing
impact: MEDIUM-HIGH
impactDescription: 10x throughput for worker queues
tags: skip-locked, queue, workers, concurrency
---
## Use SKIP LOCKED for Non-Blocking Queue Processing
When multiple workers process a queue, SKIP LOCKED allows workers to process different rows without waiting.
**Incorrect (workers block each other):**
```sql
-- Worker 1 and Worker 2 both try to get next job
begin;
select * from jobs where status = 'pending' order by created_at limit 1 for update;
-- Worker 2 waits for Worker 1's lock to release!
```
**Correct (SKIP LOCKED for parallel processing):**
```sql
-- Each worker skips locked rows and gets the next available
begin;
select * from jobs
where status = 'pending'
order by created_at
limit 1
for update skip locked;
-- Worker 1 gets job 1, Worker 2 gets job 2 (no waiting)
update jobs set status = 'processing' where id = $1;
commit;
```
Complete queue pattern:
```sql
-- Atomic claim-and-update in one statement
update jobs
set status = 'processing', worker_id = $1, started_at = now()
where id = (
select id from jobs
where status = 'pending'
order by created_at
limit 1
for update skip locked
)
returning *;
```
Reference: [SELECT FOR UPDATE SKIP LOCKED](https://www.postgresql.org/docs/current/sql-select.html#SQL-FOR-UPDATE-SHARE)
@@ -0,0 +1,45 @@
---
title: Use EXPLAIN ANALYZE to Diagnose Slow Queries
impact: LOW-MEDIUM
impactDescription: Identify exact bottlenecks in query execution
tags: explain, analyze, diagnostics, query-plan
---
## Use EXPLAIN ANALYZE to Diagnose Slow Queries
EXPLAIN ANALYZE executes the query and shows actual timings, revealing the true performance bottlenecks.
**Incorrect (guessing at performance issues):**
```sql
-- Query is slow, but why?
select * from orders where customer_id = 123 and status = 'pending';
-- "It must be missing an index" - but which one?
```
**Correct (use EXPLAIN ANALYZE):**
```sql
explain (analyze, buffers, format text)
select * from orders where customer_id = 123 and status = 'pending';
-- Output reveals the issue:
-- Seq Scan on orders (cost=0.00..25000.00 rows=50 width=100) (actual time=0.015..450.123 rows=50 loops=1)
-- Filter: ((customer_id = 123) AND (status = 'pending'::text))
-- Rows Removed by Filter: 999950
-- Buffers: shared hit=5000 read=15000
-- Planning Time: 0.150 ms
-- Execution Time: 450.500 ms
```
Key things to look for:
```sql
-- Seq Scan on large tables = missing index
-- Rows Removed by Filter = poor selectivity or missing index
-- Buffers: read >> hit = data not cached, needs more memory
-- Nested Loop with high loops = consider different join strategy
-- Sort Method: external merge = work_mem too low
```
Reference: [EXPLAIN](https://supabase.com/docs/guides/database/inspect)
@@ -0,0 +1,55 @@
---
title: Enable pg_stat_statements for Query Analysis
impact: LOW-MEDIUM
impactDescription: Identify top resource-consuming queries
tags: pg-stat-statements, monitoring, statistics, performance
---
## Enable pg_stat_statements for Query Analysis
pg_stat_statements tracks execution statistics for all queries, helping identify slow and frequent queries.
**Incorrect (no visibility into query patterns):**
```sql
-- Database is slow, but which queries are the problem?
-- No way to know without pg_stat_statements
```
**Correct (enable and query pg_stat_statements):**
```sql
-- Enable the extension
create extension if not exists pg_stat_statements;
-- Find slowest queries by total time
select
calls,
round(total_exec_time::numeric, 2) as total_time_ms,
round(mean_exec_time::numeric, 2) as mean_time_ms,
query
from pg_stat_statements
order by total_exec_time desc
limit 10;
-- Find most frequent queries
select calls, query
from pg_stat_statements
order by calls desc
limit 10;
-- Reset statistics after optimization
select pg_stat_statements_reset();
```
Key metrics to monitor:
```sql
-- Queries with high mean time (candidates for optimization)
select query, mean_exec_time, calls
from pg_stat_statements
where mean_exec_time > 100 -- > 100ms average
order by mean_exec_time desc;
```
Reference: [pg_stat_statements](https://supabase.com/docs/guides/database/extensions/pg_stat_statements)
@@ -0,0 +1,55 @@
---
title: Maintain Table Statistics with VACUUM and ANALYZE
impact: MEDIUM
impactDescription: 2-10x better query plans with accurate statistics
tags: vacuum, analyze, statistics, maintenance, autovacuum
---
## Maintain Table Statistics with VACUUM and ANALYZE
Outdated statistics cause the query planner to make poor decisions. VACUUM reclaims space, ANALYZE updates statistics.
**Incorrect (stale statistics):**
```sql
-- Table has 1M rows but stats say 1000
-- Query planner chooses wrong strategy
explain select * from orders where status = 'pending';
-- Shows: Seq Scan (because stats show small table)
-- Actually: Index Scan would be much faster
```
**Correct (maintain fresh statistics):**
```sql
-- Manually analyze after large data changes
analyze orders;
-- Analyze specific columns used in WHERE clauses
analyze orders (status, created_at);
-- Check when tables were last analyzed
select
relname,
last_vacuum,
last_autovacuum,
last_analyze,
last_autoanalyze
from pg_stat_user_tables
order by last_analyze nulls first;
```
Autovacuum tuning for busy tables:
```sql
-- Increase frequency for high-churn tables
alter table orders set (
autovacuum_vacuum_scale_factor = 0.05, -- Vacuum at 5% dead tuples (default 20%)
autovacuum_analyze_scale_factor = 0.02 -- Analyze at 2% changes (default 10%)
);
-- Check autovacuum status
select * from pg_stat_progress_vacuum;
```
Reference: [VACUUM](https://supabase.com/docs/guides/database/database-size#vacuum-operations)
@@ -0,0 +1,44 @@
---
title: Create Composite Indexes for Multi-Column Queries
impact: HIGH
impactDescription: 5-10x faster multi-column queries
tags: indexes, composite-index, multi-column, query-optimization
---
## Create Composite Indexes for Multi-Column Queries
When queries filter on multiple columns, a composite index is more efficient than separate single-column indexes.
**Incorrect (separate indexes require bitmap scan):**
```sql
-- Two separate indexes
create index orders_status_idx on orders (status);
create index orders_created_idx on orders (created_at);
-- Query must combine both indexes (slower)
select * from orders where status = 'pending' and created_at > '2024-01-01';
```
**Correct (composite index):**
```sql
-- Single composite index (leftmost column first for equality checks)
create index orders_status_created_idx on orders (status, created_at);
-- Query uses one efficient index scan
select * from orders where status = 'pending' and created_at > '2024-01-01';
```
**Column order matters** - place equality columns first, range columns last:
```sql
-- Good: status (=) before created_at (>)
create index idx on orders (status, created_at);
-- Works for: WHERE status = 'pending'
-- Works for: WHERE status = 'pending' AND created_at > '2024-01-01'
-- Does NOT work for: WHERE created_at > '2024-01-01' (leftmost prefix rule)
```
Reference: [Multicolumn Indexes](https://www.postgresql.org/docs/current/indexes-multicolumn.html)
@@ -0,0 +1,40 @@
---
title: Use Covering Indexes to Avoid Table Lookups
impact: MEDIUM-HIGH
impactDescription: 2-5x faster queries by eliminating heap fetches
tags: indexes, covering-index, include, index-only-scan
---
## Use Covering Indexes to Avoid Table Lookups
Covering indexes include all columns needed by a query, enabling index-only scans that skip the table entirely.
**Incorrect (index scan + heap fetch):**
```sql
create index users_email_idx on users (email);
-- Must fetch name and created_at from table heap
select email, name, created_at from users where email = 'user@example.com';
```
**Correct (index-only scan with INCLUDE):**
```sql
-- Include non-searchable columns in the index
create index users_email_idx on users (email) include (name, created_at);
-- All columns served from index, no table access needed
select email, name, created_at from users where email = 'user@example.com';
```
Use INCLUDE for columns you SELECT but don't filter on:
```sql
-- Searching by status, but also need customer_id and total
create index orders_status_idx on orders (status) include (customer_id, total);
select status, customer_id, total from orders where status = 'shipped';
```
Reference: [Index-Only Scans](https://www.postgresql.org/docs/current/indexes-index-only-scans.html)
@@ -0,0 +1,45 @@
---
title: Choose the Right Index Type for Your Data
impact: HIGH
impactDescription: 10-100x improvement with correct index type
tags: indexes, btree, gin, brin, hash, index-types
---
## Choose the Right Index Type for Your Data
Different index types excel at different query patterns. The default B-tree isn't always optimal.
**Incorrect (B-tree for JSONB containment):**
```sql
-- B-tree cannot optimize containment operators
create index products_attrs_idx on products (attributes);
select * from products where attributes @> '{"color": "red"}';
-- Full table scan - B-tree doesn't support @> operator
```
**Correct (GIN for JSONB):**
```sql
-- GIN supports @>, ?, ?&, ?| operators
create index products_attrs_idx on products using gin (attributes);
select * from products where attributes @> '{"color": "red"}';
```
Index type guide:
```sql
-- B-tree (default): =, <, >, BETWEEN, IN, IS NULL
create index users_created_idx on users (created_at);
-- GIN: arrays, JSONB, full-text search
create index posts_tags_idx on posts using gin (tags);
-- BRIN: large time-series tables (10-100x smaller)
create index events_time_idx on events using brin (created_at);
-- Hash: equality-only (slightly faster than B-tree for =)
create index sessions_token_idx on sessions using hash (token);
```
Reference: [Index Types](https://www.postgresql.org/docs/current/indexes-types.html)
@@ -0,0 +1,43 @@
---
title: Add Indexes on WHERE and JOIN Columns
impact: CRITICAL
impactDescription: 100-1000x faster queries on large tables
tags: indexes, performance, sequential-scan, query-optimization
---
## Add Indexes on WHERE and JOIN Columns
Queries filtering or joining on unindexed columns cause full table scans, which become exponentially slower as tables grow.
**Incorrect (sequential scan on large table):**
```sql
-- No index on customer_id causes full table scan
select * from orders where customer_id = 123;
-- EXPLAIN shows: Seq Scan on orders (cost=0.00..25000.00 rows=100 width=85)
```
**Correct (index scan):**
```sql
-- Create index on frequently filtered column
create index orders_customer_id_idx on orders (customer_id);
select * from orders where customer_id = 123;
-- EXPLAIN shows: Index Scan using orders_customer_id_idx (cost=0.42..8.44 rows=100 width=85)
```
For JOIN columns, always index the foreign key side:
```sql
-- Index the referencing column
create index orders_customer_id_idx on orders (customer_id);
select c.name, o.total
from customers c
join orders o on o.customer_id = c.id;
```
Reference: [Query Optimization](https://supabase.com/docs/guides/database/query-optimization)
@@ -0,0 +1,45 @@
---
title: Use Partial Indexes for Filtered Queries
impact: HIGH
impactDescription: 5-20x smaller indexes, faster writes and queries
tags: indexes, partial-index, query-optimization, storage
---
## Use Partial Indexes for Filtered Queries
Partial indexes only include rows matching a WHERE condition, making them smaller and faster when queries consistently filter on the same condition.
**Incorrect (full index includes irrelevant rows):**
```sql
-- Index includes all rows, even soft-deleted ones
create index users_email_idx on users (email);
-- Query always filters active users
select * from users where email = 'user@example.com' and deleted_at is null;
```
**Correct (partial index matches query filter):**
```sql
-- Index only includes active users
create index users_active_email_idx on users (email)
where deleted_at is null;
-- Query uses the smaller, faster index
select * from users where email = 'user@example.com' and deleted_at is null;
```
Common use cases for partial indexes:
```sql
-- Only pending orders (status rarely changes once completed)
create index orders_pending_idx on orders (created_at)
where status = 'pending';
-- Only non-null values
create index products_sku_idx on products (sku)
where sku is not null;
```
Reference: [Partial Indexes](https://www.postgresql.org/docs/current/indexes-partial.html)
@@ -0,0 +1,46 @@
---
title: Choose Appropriate Data Types
impact: HIGH
impactDescription: 50% storage reduction, faster comparisons
tags: data-types, schema, storage, performance
---
## Choose Appropriate Data Types
Using the right data types reduces storage, improves query performance, and prevents bugs.
**Incorrect (wrong data types):**
```sql
create table users (
id int, -- Will overflow at 2.1 billion
email varchar(255), -- Unnecessary length limit
created_at timestamp, -- Missing timezone info
is_active varchar(5), -- String for boolean
price varchar(20) -- String for numeric
);
```
**Correct (appropriate data types):**
```sql
create table users (
id bigint generated always as identity primary key, -- 9 quintillion max
email text, -- No artificial limit, same performance as varchar
created_at timestamptz, -- Always store timezone-aware timestamps
is_active boolean default true, -- 1 byte vs variable string length
price numeric(10,2) -- Exact decimal arithmetic
);
```
Key guidelines:
```sql
-- IDs: use bigint, not int (future-proofing)
-- Strings: use text, not varchar(n) unless constraint needed
-- Time: use timestamptz, not timestamp
-- Money: use numeric, not float (precision matters)
-- Enums: use text with check constraint or create enum type
```
Reference: [Data Types](https://www.postgresql.org/docs/current/datatype.html)
@@ -0,0 +1,59 @@
---
title: Index Foreign Key Columns
impact: HIGH
impactDescription: 10-100x faster JOINs and CASCADE operations
tags: foreign-key, indexes, joins, schema
---
## Index Foreign Key Columns
Postgres does not automatically index foreign key columns. Missing indexes cause slow JOINs and CASCADE operations.
**Incorrect (unindexed foreign key):**
```sql
create table orders (
id bigint generated always as identity primary key,
customer_id bigint references customers(id) on delete cascade,
total numeric(10,2)
);
-- No index on customer_id!
-- JOINs and ON DELETE CASCADE both require full table scan
select * from orders where customer_id = 123; -- Seq Scan
delete from customers where id = 123; -- Locks table, scans all orders
```
**Correct (indexed foreign key):**
```sql
create table orders (
id bigint generated always as identity primary key,
customer_id bigint references customers(id) on delete cascade,
total numeric(10,2)
);
-- Always index the FK column
create index orders_customer_id_idx on orders (customer_id);
-- Now JOINs and cascades are fast
select * from orders where customer_id = 123; -- Index Scan
delete from customers where id = 123; -- Uses index, fast cascade
```
Find missing FK indexes:
```sql
select
conrelid::regclass as table_name,
a.attname as fk_column
from pg_constraint c
join pg_attribute a on a.attrelid = c.conrelid and a.attnum = any(c.conkey)
where c.contype = 'f'
and not exists (
select 1 from pg_index i
where i.indrelid = c.conrelid and a.attnum = any(i.indkey)
);
```
Reference: [Foreign Keys](https://www.postgresql.org/docs/current/ddl-constraints.html#DDL-CONSTRAINTS-FK)
@@ -0,0 +1,55 @@
---
title: Use Lowercase Identifiers for Compatibility
impact: MEDIUM
impactDescription: Avoid case-sensitivity bugs with tools, ORMs, and AI assistants
tags: naming, identifiers, case-sensitivity, schema, conventions
---
## Use Lowercase Identifiers for Compatibility
PostgreSQL folds unquoted identifiers to lowercase. Quoted mixed-case identifiers require quotes forever and cause issues with tools, ORMs, and AI assistants that may not recognize them.
**Incorrect (mixed-case identifiers):**
```sql
-- Quoted identifiers preserve case but require quotes everywhere
CREATE TABLE "Users" (
"userId" bigint PRIMARY KEY,
"firstName" text,
"lastName" text
);
-- Must always quote or queries fail
SELECT "firstName" FROM "Users" WHERE "userId" = 1;
-- This fails - Users becomes users without quotes
SELECT firstName FROM Users;
-- ERROR: relation "users" does not exist
```
**Correct (lowercase snake_case):**
```sql
-- Unquoted lowercase identifiers are portable and tool-friendly
CREATE TABLE users (
user_id bigint PRIMARY KEY,
first_name text,
last_name text
);
-- Works without quotes, recognized by all tools
SELECT first_name FROM users WHERE user_id = 1;
```
Common sources of mixed-case identifiers:
```sql
-- ORMs often generate quoted camelCase - configure them to use snake_case
-- Migrations from other databases may preserve original casing
-- Some GUI tools quote identifiers by default - disable this
-- If stuck with mixed-case, create views as a compatibility layer
CREATE VIEW users AS SELECT "userId" AS user_id, "firstName" AS first_name FROM "Users";
```
Reference: [Identifiers and Key Words](https://www.postgresql.org/docs/current/sql-syntax-lexical.html#SQL-SYNTAX-IDENTIFIERS)
@@ -0,0 +1,55 @@
---
title: Partition Large Tables for Better Performance
impact: MEDIUM-HIGH
impactDescription: 5-20x faster queries and maintenance on large tables
tags: partitioning, large-tables, time-series, performance
---
## Partition Large Tables for Better Performance
Partitioning splits a large table into smaller pieces, improving query performance and maintenance operations.
**Incorrect (single large table):**
```sql
create table events (
id bigint generated always as identity,
created_at timestamptz,
data jsonb
);
-- 500M rows, queries scan everything
select * from events where created_at > '2024-01-01'; -- Slow
vacuum events; -- Takes hours, locks table
```
**Correct (partitioned by time range):**
```sql
create table events (
id bigint generated always as identity,
created_at timestamptz not null,
data jsonb
) partition by range (created_at);
-- Create partitions for each month
create table events_2024_01 partition of events
for values from ('2024-01-01') to ('2024-02-01');
create table events_2024_02 partition of events
for values from ('2024-02-01') to ('2024-03-01');
-- Queries only scan relevant partitions
select * from events where created_at > '2024-01-15'; -- Only scans events_2024_01+
-- Drop old data instantly
drop table events_2023_01; -- Instant vs DELETE taking hours
```
When to partition:
- Tables > 100M rows
- Time-series data with date-based queries
- Need to efficiently drop old data
Reference: [Table Partitioning](https://www.postgresql.org/docs/current/ddl-partitioning.html)
@@ -0,0 +1,61 @@
---
title: Select Optimal Primary Key Strategy
impact: HIGH
impactDescription: Better index locality, reduced fragmentation
tags: primary-key, identity, uuid, serial, schema
---
## Select Optimal Primary Key Strategy
Primary key choice affects insert performance, index size, and replication
efficiency.
**Incorrect (problematic PK choices):**
```sql
-- identity is the SQL-standard approach
create table users (
id serial primary key -- Works, but IDENTITY is recommended
);
-- Random UUIDs (v4) cause index fragmentation
create table orders (
id uuid default gen_random_uuid() primary key -- UUIDv4 = random = scattered inserts
);
```
**Correct (optimal PK strategies):**
```sql
-- Use IDENTITY for sequential IDs (SQL-standard, best for most cases)
create table users (
id bigint generated always as identity primary key
);
-- For distributed systems needing UUIDs, use UUIDv7 (time-ordered)
-- Requires pg_uuidv7 extension: create extension pg_uuidv7;
create table orders (
id uuid default uuid_generate_v7() primary key -- Time-ordered, no fragmentation
);
-- Alternative: time-prefixed IDs for sortable, distributed IDs (no extension needed)
create table events (
id text default concat(
to_char(now() at time zone 'utc', 'YYYYMMDDHH24MISSMS'),
gen_random_uuid()::text
) primary key
);
```
Guidelines:
- Single database: `bigint identity` (sequential, 8 bytes, SQL-standard)
- Distributed/exposed IDs: UUIDv7 (requires pg_uuidv7) or ULID (time-ordered, no
fragmentation)
- `serial` works but `identity` is SQL-standard and preferred for new
applications
- Avoid random UUIDs (v4) as primary keys on large tables (causes index
fragmentation)
Reference:
[Identity Columns](https://www.postgresql.org/docs/current/sql-createtable.html#SQL-CREATETABLE-PARMS-GENERATED-IDENTITY)
@@ -0,0 +1,54 @@
---
title: Apply Principle of Least Privilege
impact: MEDIUM
impactDescription: Reduced attack surface, better audit trail
tags: privileges, security, roles, permissions
---
## Apply Principle of Least Privilege
Grant only the minimum permissions required. Never use superuser for application queries.
**Incorrect (overly broad permissions):**
```sql
-- Application uses superuser connection
-- Or grants ALL to application role
grant all privileges on all tables in schema public to app_user;
grant all privileges on all sequences in schema public to app_user;
-- Any SQL injection becomes catastrophic
-- drop table users; cascades to everything
```
**Correct (minimal, specific grants):**
```sql
-- Create role with no default privileges
create role app_readonly nologin;
-- Grant only SELECT on specific tables
grant usage on schema public to app_readonly;
grant select on public.products, public.categories to app_readonly;
-- Create role for writes with limited scope
create role app_writer nologin;
grant usage on schema public to app_writer;
grant select, insert, update on public.orders to app_writer;
grant usage on sequence orders_id_seq to app_writer;
-- No DELETE permission
-- Login role inherits from these
create role app_user login password 'xxx';
grant app_writer to app_user;
```
Revoke public defaults:
```sql
-- Revoke default public access
revoke all on schema public from public;
revoke all on all tables in schema public from public;
```
Reference: [Roles and Privileges](https://supabase.com/blog/postgres-roles-and-privileges)
@@ -0,0 +1,50 @@
---
title: Enable Row Level Security for Multi-Tenant Data
impact: CRITICAL
impactDescription: Database-enforced tenant isolation, prevent data leaks
tags: rls, row-level-security, multi-tenant, security
---
## Enable Row Level Security for Multi-Tenant Data
Row Level Security (RLS) enforces data access at the database level, ensuring users only see their own data.
**Incorrect (application-level filtering only):**
```sql
-- Relying only on application to filter
select * from orders where user_id = $current_user_id;
-- Bug or bypass means all data is exposed!
select * from orders; -- Returns ALL orders
```
**Correct (database-enforced RLS):**
```sql
-- Enable RLS on the table
alter table orders enable row level security;
-- Create policy for users to see only their orders
create policy orders_user_policy on orders
for all
using (user_id = current_setting('app.current_user_id')::bigint);
-- Force RLS even for table owners
alter table orders force row level security;
-- Set user context and query
set app.current_user_id = '123';
select * from orders; -- Only returns orders for user 123
```
Policy for authenticated role:
```sql
create policy orders_user_policy on orders
for all
to authenticated
using (user_id = auth.uid());
```
Reference: [Row Level Security](https://supabase.com/docs/guides/database/postgres/row-level-security)
@@ -0,0 +1,57 @@
---
title: Optimize RLS Policies for Performance
impact: HIGH
impactDescription: 5-10x faster RLS queries with proper patterns
tags: rls, performance, security, optimization
---
## Optimize RLS Policies for Performance
Poorly written RLS policies can cause severe performance issues. Use subqueries and indexes strategically.
**Incorrect (function called for every row):**
```sql
create policy orders_policy on orders
using (auth.uid() = user_id); -- auth.uid() called per row!
-- With 1M rows, auth.uid() is called 1M times
```
**Correct (wrap functions in SELECT):**
```sql
create policy orders_policy on orders
using ((select auth.uid()) = user_id); -- Called once, cached
-- 100x+ faster on large tables
```
Use security definer functions for complex checks:
```sql
-- Create helper function (runs as definer, bypasses RLS)
create or replace function is_team_member(team_id bigint)
returns boolean
language sql
security definer
set search_path = ''
as $$
select exists (
select 1 from public.team_members
where team_id = $1 and user_id = (select auth.uid())
);
$$;
-- Use in policy (indexed lookup, not per-row check)
create policy team_orders_policy on orders
using ((select is_team_member(team_id)));
```
Always add indexes on columns used in RLS policies:
```sql
create index orders_user_id_idx on orders (user_id);
```
Reference: [RLS Performance](https://supabase.com/docs/guides/database/postgres/row-level-security#rls-performance-recommendations)
@@ -0,0 +1,72 @@
---
name: using-neon
description: Guides and best practices for working with Neon Serverless Postgres. Covers getting started, local development with Neon, choosing a connection method, Neon features, authentication (@neondatabase/auth), PostgREST-style data API (@neondatabase/neon-js), Neon CLI, and Neon's Platform API/SDKs. Use for any Neon-related questions.
---
# Neon Serverless Postgres
Neon is a serverless Postgres platform that separates compute and storage to offer autoscaling, branching, instant restore, and scale-to-zero. It's fully compatible with Postgres and works with any language, framework, or ORM that supports Postgres.
## Neon Documentation
Always reference the Neon documentation before making Neon-related claims. The documentation is the source of truth for all Neon-related information.
Below you'll find a list of resources organized by area of concern. This is meant to support you find the right documentation pages to fetch and add a bit of additonal context.
You can use the `curl` commands to fetch the documentation page as markdown:
**Documentation:**
```bash
# Get list of all Neon docs
curl https://neon.tech/llms.txt
# Fetch any doc page as markdown
curl -H "Accept: text/markdown" https://neon.tech/docs/<path>
```
Don't guess docs pages. Use the `llms.txt` index to find the relevant URL or follow the links in the resources below.
## Overview of Resources
Reference the appropriate resource file based on the user's needs:
### Core Guides
| Area | Resource | When to Use |
| ------------------ | ---------------------------------- | -------------------------------------------------------------- |
| What is Neon | `references/what-is-neon.md` | Understanding Neon concepts, architecture, core resources |
| Referencing Docs | `references/referencing-docs.md` | Looking up official documentation, verifying information |
| Features | `references/features.md` | Branching, autoscaling, scale-to-zero, instant restore |
| Getting Started | `references/getting-started.md` | Setting up a project, connection strings, dependencies, schema |
| Connection Methods | `references/connection-methods.md` | Choosing drivers based on platform and runtime |
| Developer Tools | `references/devtools.md` | VSCode extension, MCP server, Neon CLI (`neon init`) |
### Database Drivers & ORMs
HTTP/WebSocket queries for serverless/edge functions.
| Area | Resource | When to Use |
| ----------------- | ------------------------------- | --------------------------------------------------- |
| Serverless Driver | `references/neon-serverless.md` | `@neondatabase/serverless` - HTTP/WebSocket queries |
| Drizzle ORM | `references/neon-drizzle.md` | Drizzle ORM integration with Neon |
### Auth & Data API SDKs
Authentication and PostgREST-style data API for Neon.
| Area | Resource | When to Use |
| ----------- | ------------------------- | ------------------------------------------------------------------- |
| Neon Auth | `references/neon-auth.md` | `@neondatabase/auth` - Authentication only |
| Neon JS SDK | `references/neon-js.md` | `@neondatabase/neon-js` - Auth + Data API (PostgREST-style queries) |
### Neon Platform API & CLI
Managing Neon resources programmatically via REST API, SDKs, or CLI.
| Area | Resource | When to Use |
| --------------------- | ----------------------------------- | -------------------------------------------- |
| Platform API Overview | `references/neon-platform-api.md` | Managing Neon resources via REST API |
| Neon CLI | `references/neon-cli.md` | Terminal workflows, scripts, CI/CD pipelines |
| TypeScript SDK | `references/neon-typescript-sdk.md` | `@neondatabase/api-client` |
| Python SDK | `references/neon-python-sdk.md` | `neon-api` package |
@@ -0,0 +1,193 @@
# Connection Methods
Guide to selecting the optimal connection method for your Neon Postgres database based on deployment platform and runtime environment.
For official documentation:
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/connect/choose-connection
```
## Decision Tree
Follow this flow to determine the right connection approach:
### 1. What Language Are You Using?
**Not TypeScript/JavaScript** → Use **TCP with connection pooling** from a secure server.
For non-TypeScript languages, connect from a secure backend server using your language's native Postgres driver with connection pooling enabled.
| Language/Framework | Documentation |
| ------------------- | ------------------------------------------- |
| Django (Python) | https://neon.tech/docs/guides/django |
| SQLAlchemy (Python) | https://neon.tech/docs/guides/sqlalchemy |
| Elixir Ecto | https://neon.tech/docs/guides/elixir-ecto |
| Laravel (PHP) | https://neon.tech/docs/guides/laravel |
| Ruby on Rails | https://neon.tech/docs/guides/ruby-on-rails |
| Go | https://neon.tech/docs/guides/go |
| Rust | https://neon.tech/docs/guides/rust |
| Java | https://neon.tech/docs/guides/java |
**TypeScript/JavaScript** → Continue to step 2.
---
### 2. Client-Side App Without Backend?
**Yes** → Use **Neon Data API** via `@neondatabase/neon-js`
This is the only option for client-side apps since browsers cannot make direct TCP connections to Postgres. See `neon-js.md` for setup.
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/reference/javascript-sdk
```
**No** → Continue to step 3.
---
### 3. Long-Running Server? (Railway, Render, traditional VPS)
**Yes** → Use **TCP with connection pooling** via `node-postgres`, `postgres.js`, or `bun:pg`
Long-running servers maintain persistent connections, so standard TCP drivers with pooling are optimal.
**No** → Continue to step 4.
---
### 4. Edge Environment Without TCP Support?
Some edge runtimes don't support TCP connections. Rarely the case anymore.
**Yes** → Continue to step 5 to check transaction requirements.
**No** → Continue to step 6 to check pooling support.
---
### 5. Does Your App Use SQL Transactions?
**Yes** → Use **WebSocket transport** via `@neondatabase/serverless` with `Pool`
WebSocket maintains connection state needed for transactions. See `neon-serverless.md` for setup.
**No** → Use **HTTP transport** via `@neondatabase/serverless`
HTTP is faster for single queries (~3 roundtrips vs ~8 for TCP). See `neon-serverless.md` for setup.
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/serverless/serverless-driver
```
---
### 6. Serverless Environment With Connection Pooling Support?
**Vercel (Fluid Compute)** → Use **TCP with `@vercel/functions`**
Vercel's Fluid compute supports connection pooling. Use `attachDatabasePool` for optimal connection management.
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/guides/vercel-connection-methods
```
**Cloudflare (with Hyperdrive)** → Use **TCP via Hyperdrive**
Cloudflare Hyperdrive provides connection pooling for Workers. Use `node-postgres` or any native TCP driver.
See https://neon.tech/docs/guides/cloudflare-hyperdrive for more on connecting with Cloudflare Workers and Hyperdrive.
**No pooling support (Netlify, Deno Deploy)** → Use `@neondatabase/serverless`
Fall back to the decision in step 5 based on transaction requirements.
---
## Quick Reference Table
| Platform | TCP Support | Pooling | Recommended Driver |
| ----------------------- | ----------- | ------------------- | -------------------------- |
| Vercel (Fluid) | Yes | `@vercel/functions` | `pg` (node-postgres) |
| Cloudflare (Hyperdrive) | Yes | Hyperdrive | `pg` (node-postgres) |
| Cloudflare Workers | No | No | `@neondatabase/serverless` |
| Netlify Functions | No | No | `@neondatabase/serverless` |
| Deno Deploy | No | No | `@neondatabase/serverless` |
| Railway / Render | Yes | Built-in | `pg` (node-postgres) |
| Client-side (browser) | No | N/A | `@neondatabase/neon-js` |
---
## ORM Support
Popular TypeScript/JavaScript ORMs all work with Neon:
| ORM | Drivers Supported | Documentation |
| ------- | ----------------------------------------------- | ------------------------------------- |
| Drizzle | `pg`, `postgres.js`, `@neondatabase/serverless` | https://neon.tech/docs/guides/drizzle |
| Kysely | `pg`, `postgres.js`, `@neondatabase/serverless` | https://neon.tech/docs/guides/kysely |
| Prisma | `pg`, `@neondatabase/serverless` | https://neon.tech/docs/guides/prisma |
| TypeORM | `pg` | https://neon.tech/docs/guides/typeorm |
All ORMs support both TCP drivers and Neon's serverless driver depending on your platform.
For Drizzle ORM integration with Neon, see `neon-drizzle.md`.
---
## Vercel Fluid + Drizzle Example
Complete database client setup for Vercel with Drizzle ORM and connection pooling. See `neon-drizzle.md` for more examples.
```typescript
// src/lib/db/client.ts
import { attachDatabasePool } from "@vercel/functions";
import { drizzle } from "drizzle-orm/node-postgres";
import { Pool } from "pg";
import * as schema from "./schema";
const pool = new Pool({
connectionString: process.env.DATABASE_URL,
});
attachDatabasePool(pool);
export const db = drizzle({ client: pool, schema });
```
**Why `attachDatabasePool`?**
- First request establishes the TCP connection (~8 roundtrips)
- Subsequent requests reuse the connection instantly
- Ensures idle connections close gracefully before function suspension
- Prevents connection leaks in serverless environments
---
## Gathering Requirements
When helping a user choose their connection method, gather this information:
1. **Deployment platform**: Where will the app run? (Vercel, Cloudflare, Netlify, Railway, browser, etc.)
2. **Runtime type**: Serverless functions, edge functions, or long-running server?
3. **Transaction requirements**: Does the app need SQL transactions?
4. **ORM preference**: Using Drizzle, Kysely, Prisma, or raw SQL?
Then provide:
- The recommended driver/package
- A working code example for their setup
- The correct npm install command
---
## Documentation Resources
| Topic | URL |
| -------------------------- | ------------------------------------------------------- |
| Choosing Connection Method | https://neon.tech/docs/connect/choose-connection |
| Serverless Driver | https://neon.tech/docs/serverless/serverless-driver |
| JavaScript SDK | https://neon.tech/docs/reference/javascript-sdk |
| Connection Pooling | https://neon.tech/docs/connect/connection-pooling |
| Vercel Connection Methods | https://neon.tech/docs/guides/vercel-connection-methods |
@@ -0,0 +1,121 @@
# Neon Developer Tools
Neon provides developer tools to enhance your local development workflow, including a VSCode extension and MCP server for AI-assisted development.
## Quick Setup with neon init
The fastest way to set up all Neon developer tools:
```bash
npx neon init
```
This command:
- Installs the Neon VSCode extension
- Configures the Neon MCP server for AI assistants
- Sets up your local environment for Neon development
For full CLI reference:
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/reference/cli-init
```
## VSCode Extension
The Neon VSCode extension provides:
- **Database Explorer**: Browse projects, branches, tables, and data
- **SQL Editor**: Write and execute queries with IntelliSense
- **Branch Management**: Create, switch, and manage database branches
- **Connection String Access**: Quick copy of connection strings
**Install from VSCode:**
1. Open Extensions (Cmd/Ctrl+Shift+X)
2. Search "Neon"
3. Install "Neon" by Neon
**Or via command line:**
```bash
code --install-extension neon.neon-vscode
```
For detailed documentation:
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/local/vscode-extension
```
## Neon MCP Server
The Neon MCP (Model Context Protocol) server enables AI assistants like Claude, Cursor, and GitHub Copilot to interact with your Neon databases directly.
### Capabilities
The MCP server provides AI assistants with:
- **Project Management**: List, create, describe, and delete projects
- **Branch Operations**: Create branches, compare schemas, reset from parent
- **SQL Execution**: Run queries and transactions
- **Schema Operations**: Describe tables, get database structure
- **Migrations**: Prepare and complete database migrations with safety checks
- **Query Tuning**: Analyze and optimize slow queries
- **Neon Auth**: Provision authentication for your branches
### Setup
**Option 1: Via neon init (Recommended)**
```bash
npx neon init
```
**Option 2: Manual Configuration**
Add to your AI assistant's MCP configuration:
```json
{
"mcpServers": {
"neon": {
"command": "npx",
"args": ["-y", "@neondatabase/mcp-server-neon"],
"env": {
"NEON_API_KEY": "your-api-key"
}
}
}
}
```
Get your API key from: https://console.neon.tech/app/settings/api-keys
### Common MCP Operations
| Operation | What It Does |
| ---------------------------- | ----------------------------- |
| `list_projects` | Show all Neon projects |
| `create_project` | Create a new project |
| `run_sql` | Execute SQL queries |
| `get_connection_string` | Get database connection URL |
| `create_branch` | Create a database branch |
| `prepare_database_migration` | Safely prepare schema changes |
| `provision_neon_auth` | Set up Neon Auth |
For full MCP server documentation:
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/ai/neon-mcp-server
```
## Documentation Resources
| Topic | URL |
| ------------------ | --------------------------------------------- |
| CLI Init Command | https://neon.tech/docs/reference/cli-init |
| VSCode Extension | https://neon.tech/docs/local/vscode-extension |
| MCP Server | https://neon.tech/docs/ai/neon-mcp-server |
| Neon CLI Reference | https://neon.tech/docs/reference/neon-cli |
@@ -0,0 +1,152 @@
# Neon Features
Overview of Neon's key platform features. For detailed information, fetch the official docs.
## Branching
Create instant, copy-on-write clones of your database at any point in time. Branches are isolated environments perfect for development, testing, and preview deployments.
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/introduction/branching
```
**Key Points:**
- Branches are instant (no data copying)
- Copy-on-write means branches only store changes from parent
- Use for: dev environments, staging, testing, preview deployments
- Branches can have their own compute endpoint
**Use Cases:**
| Use Case | Description |
| ------------------- | ------------------------------------------- |
| Development | Each developer gets isolated branch |
| Preview Deployments | Branch per PR/preview URL |
| Testing | Reset test data by recreating branch |
| Schema Migrations | Test migrations on branch before production |
If the Neon MCP server is available, you can use it to list and create branches. Otherwise, refer to the Neon CLI or Platform API.
## Autoscaling
Neon automatically scales compute resources based on workload demand.
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/introduction/autoscaling
```
**Key Points:**
- Scales between min and max compute units (CUs)
- Responds to CPU and memory pressure
- No manual intervention required
- Configure limits per project or endpoint
## Scale to Zero
Databases automatically suspend after a period of inactivity, reducing costs to storage-only.
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/introduction/scale-to-zero
```
**Key Points:**
- Default suspend after 5 minutes of inactivity (configurable)
- First query after suspend has ~500ms cold start
- Storage is always maintained
- Perfect for dev/staging environments with intermittent use
## Instant Restore
Restore your database to any point within your retention window without backups.
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/introduction/branch-restore
```
**Key Points:**
- Point-in-time recovery without pre-configured backups
- Restore window depends on plan (7-30 days)
- Create branches from any point in history
- Time Travel queries to view historical data
## Read Replicas
Create read-only compute endpoints to scale read workloads.
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/introduction/read-replicas
```
**Key Points:**
- Read replicas share storage with primary (no data duplication)
- Instant creation
- Independent scaling from primary
- Use for: analytics, reporting, read-heavy workloads
## Connection Pooling
Built-in connection pooling via PgBouncer for efficient connection management.
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/connect/connection-pooling
```
**Key Points:**
- Enabled by adding `-pooler` to endpoint hostname
- Transaction mode by default
- Supports up to 10,000 concurrent connections
- Essential for serverless environments
## IP Allow Lists
Restrict database access to specific IP addresses or ranges.
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/introduction/ip-allow
```
## Logical Replication
Replicate data to/from external Postgres databases.
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/guides/logical-replication-guide
```
## Neon Auth
Managed authentication that branches with your database.
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/auth/overview
```
**Key Points:**
- Sign-in/sign-up with email, social providers (Google, GitHub)
- Session management
- UI components included
- Branches with your database
For setup, see `neon-auth.md`. For auth + data API, see `neon-js.md`.
## Feature Documentation Reference
| Feature | Documentation | Resource |
| ------------------- | ------------------------------------------------------- | -------------- |
| Branching | https://neon.tech/docs/introduction/branching | - |
| Autoscaling | https://neon.tech/docs/introduction/autoscaling | - |
| Scale to Zero | https://neon.tech/docs/introduction/scale-to-zero | - |
| Instant Restore | https://neon.tech/docs/introduction/branch-restore | - |
| Read Replicas | https://neon.tech/docs/introduction/read-replicas | - |
| Connection Pooling | https://neon.tech/docs/connect/connection-pooling | - |
| IP Allow | https://neon.tech/docs/introduction/ip-allow | - |
| Logical Replication | https://neon.tech/docs/guides/logical-replication-guide | - |
| Neon Auth | https://neon.tech/docs/auth/overview | `neon-auth.md` |
| Data API | https://neon.tech/docs/data-api/overview | `neon-js.md` |
@@ -0,0 +1,183 @@
# Getting Started with Neon
Interactive guide to help users get started with Neon in their project. Sets up their Neon project (with a connection string) and connects their database to their code.
For the official getting started guide:
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/get-started/signing-up
```
## Interactive Setup Flow
### Step 1: Check Organizations and Projects
**First, check for organizations:**
- If they have 1 organization: Default to that organization
- If they have multiple organizations: List all and ask which one to use
**Then, check for projects within the selected organization:**
- **No projects**: Ask if they want to create a new project
- **1 project**: Ask "Would you like to use '{project_name}' or create a new one?"
- **Multiple projects (<6)**: List all and let them choose
- **Many projects (6+)**: List recent projects, offer to create new or specify by name/ID
### Step 2: Database Setup
**Get the connection string:**
- Use the MCP server to get the connection string for the selected project
**Configure it for their environment:**
- Most projects use a `.env` file with `DATABASE_URL`
- For other setups, check project structure and ask
**Before modifying .env:**
1. Try to read the .env file first
2. If readable: Use search_replace to update or append
3. If unreadable: Use append command or show the line to add manually:
```
DATABASE_URL=postgresql://user:password@host/database
```
### Step 3: Install Dependencies
Recommend drivers based on deployment platform and runtime. For detailed guidance, see `connection-methods.md`.
**Quick Recommendations:**
| Environment | Driver | Install |
| ------------------------ | -------------------------- | -------------------------------------- |
| Vercel (Edge/Serverless) | `@neondatabase/serverless` | `npm install @neondatabase/serverless` |
| Cloudflare Workers | `@neondatabase/serverless` | `npm install @neondatabase/serverless` |
| AWS Lambda | `@neondatabase/serverless` | `npm install @neondatabase/serverless` |
| Traditional Node.js | `pg` | `npm install pg` |
| Long-running servers | `pg` with pooling | `npm install pg` |
For detailed serverless driver usage, see `neon-serverless.md`.
For complex scenarios (multiple runtimes, hybrid architectures), reference `connection-methods.md`.
### Step 4: Understand the Project
**If it's an empty/new project:**
Ask briefly (1-2 questions):
- What are they building?
- Any specific technologies?
**If it's an established project:**
Skip questions - infer from codebase. Update relevant code to use the driver.
### Step 5: Authentication (Optional)
**Skip if project doesn't need auth** (CLI tools, scripts, static sites).
**If project could benefit from auth:**
Ask: "Does your app need user authentication? Neon Auth can handle sign-in/sign-up, social login, and session management."
**If they want auth:**
- Use MCP server `provision_neon_auth` tool
- Guide through framework-specific setup
- Configure environment variables
- Set up basic auth code
For detailed auth setup, see `neon-auth.md`. For auth + database queries, see `neon-js.md`.
### Step 6: ORM Setup
**Check for existing ORM** (Prisma, Drizzle, TypeORM).
**If no ORM found:**
Ask: "Want to set up an ORM for type-safe database queries?"
If yes, suggest based on project. If no, proceed with raw SQL.
For Drizzle ORM integration, see `neon-drizzle.md`.
### Step 7: Schema Setup
**Check for existing schema:**
- SQL migration files
- ORM schemas (Prisma, Drizzle)
- Database initialization scripts
**If existing schema found:**
Ask: "Found existing schema definitions. Want to migrate these to your Neon database?"
**If no schema:**
Ask if they want to:
1. Create a simple example schema (users table)
2. Design a custom schema together
3. Skip schema setup for now
**Example schema:**
```sql
CREATE TABLE users (
id SERIAL PRIMARY KEY,
email VARCHAR(255) UNIQUE NOT NULL,
name VARCHAR(255),
created_at TIMESTAMP DEFAULT NOW()
);
```
### Step 8: What's Next
"You're all set! Here are some things I can help with:
- Neon-specific features (branching, autoscaling, scale-to-zero)
- Connection pooling for production
- Writing queries or building API endpoints
- Database migrations and schema changes
- Performance optimization"
## Security Best Practices
1. Never commit connection strings to version control
2. Use environment variables for all credentials
3. Prefer SSL connections (default in Neon)
4. Use least-privilege database roles
5. Rotate API keys and passwords regularly
## Resume Support
If user says "Continue with Neon setup", check what's already configured:
- MCP server connection
- .env file with DATABASE_URL
- Dependencies installed
- Schema created
Then resume from where they left off.
## Developer Tools
For the best development experience, set up Neon's developer tools:
```bash
npx neon init
```
This installs the VSCode extension and configures the MCP server for AI-assisted development.
For detailed setup instructions, see `devtools.md`.
## Documentation Resources
| Topic | URL |
| ------------------ | --------------------------------------------------- |
| Getting Started | https://neon.tech/docs/get-started/signing-up |
| Connecting to Neon | https://neon.tech/docs/connect/connect-intro |
| Connection String | https://neon.tech/docs/connect/connect-from-any-app |
| Frameworks Guide | https://neon.tech/docs/get-started/frameworks |
| ORMs Guide | https://neon.tech/docs/get-started/orms |
| VSCode Extension | https://neon.tech/docs/local/vscode-extension |
| MCP Server | https://neon.tech/docs/ai/neon-mcp-server |
@@ -0,0 +1,141 @@
# Neon Auth
Neon Auth provides authentication for your application. It's available as:
- `@neondatabase/auth` - Auth only (smaller bundle)
- `@neondatabase/neon-js` - Auth + Data API (full SDK, see `neon-js.md`)
For official documentation:
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/auth/overview
```
## Package Selection
| Need | Package | Bundle |
| ----------------------- | ----------------------- | ------- |
| Auth only | `@neondatabase/auth` | Smaller |
| Auth + Database queries | `@neondatabase/neon-js` | Full |
## Installation
```bash
# Auth only
npm install @neondatabase/auth
# Auth + Data API
npm install @neondatabase/neon-js
```
## Quick Setup Patterns
### Next.js App Router
**1. API Route Handler:**
```typescript
// app/api/auth/[...path]/route.ts
import { authApiHandler } from "@neondatabase/auth/next";
export const { GET, POST } = authApiHandler();
```
**2. Auth Client:**
```typescript
// lib/auth/client.ts
import { createAuthClient } from "@neondatabase/auth/next";
export const authClient = createAuthClient();
```
**3. Use in Components:**
```typescript
"use client";
import { authClient } from "@/lib/auth/client";
function AuthStatus() {
const session = authClient.useSession();
if (session.isPending) return <div>Loading...</div>;
if (!session.data) return <SignInButton />;
return <div>Hello, {session.data.user.name}</div>;
}
```
### React SPA
```typescript
import { createAuthClient } from "@neondatabase/auth";
import { BetterAuthReactAdapter } from "@neondatabase/auth/react/adapters";
const authClient = createAuthClient(import.meta.env.VITE_NEON_AUTH_URL, {
adapter: BetterAuthReactAdapter(),
});
```
### Node.js Backend
```typescript
import { createAuthClient } from "@neondatabase/auth";
const auth = createAuthClient(process.env.NEON_AUTH_URL!);
await auth.signIn.email({ email, password });
const session = await auth.getSession();
```
## Environment Variables
```bash
# Next.js (.env.local)
NEON_AUTH_BASE_URL=https://ep-xxx.neonauth.c-2.us-east-2.aws.neon.build/dbname/auth
NEXT_PUBLIC_NEON_AUTH_URL=https://ep-xxx.neonauth.c-2.us-east-2.aws.neon.build/dbname/auth
# Vite/React (.env)
VITE_NEON_AUTH_URL=https://ep-xxx.neonauth.c-2.us-east-2.aws.neon.build/dbname/auth
```
## Sub-Resources
For detailed documentation:
| Topic | Resource |
| ------------------------ | ------------------------------ |
| Next.js App Router setup | `neon-auth/setup-nextjs.md` |
| React SPA setup | `neon-auth/setup-react-spa.md` |
| Auth methods reference | `neon-auth/auth-methods.md` |
| UI components | `neon-auth/ui-components.md` |
| Common mistakes | `neon-auth/common-mistakes.md` |
## Key Imports
```typescript
// Auth client (Next.js)
import { authApiHandler, createAuthClient } from "@neondatabase/auth/next";
// Auth client (vanilla)
import { createAuthClient } from "@neondatabase/auth";
// React adapter (NOT from main entry)
import { BetterAuthReactAdapter } from "@neondatabase/auth/react/adapters";
// UI components
import {
NeonAuthUIProvider,
AuthView,
SignInForm,
} from "@neondatabase/auth/react/ui";
import { authViewPaths } from "@neondatabase/auth/react/ui/server";
// CSS
import "@neondatabase/auth/ui/css";
```
## Common Mistakes
1. **Wrong adapter import**: Import `BetterAuthReactAdapter` from `auth/react/adapters` subpath
2. **Forgetting to call adapter**: Use `BetterAuthReactAdapter()` with parentheses
3. **Missing CSS**: Import from `ui/css` or `ui/tailwind` (not both)
4. **Missing "use client"**: Required for components using `useSession()`
5. **Wrong createAuthClient signature**: First arg is URL: `createAuthClient(url, { adapter })`
See `neon-auth/common-mistakes.md` for detailed examples.
@@ -0,0 +1,241 @@
# Neon Auth - Auth Methods Reference
Complete reference for authentication methods, session management, and error handling.
## Auth Methods
### Sign Up
```typescript
await auth.signUp.email({
email: "user@example.com",
password: "securepassword",
name: "John Doe", // Optional
});
```
### Sign In
```typescript
// Email/password
await auth.signIn.email({
email: "user@example.com",
password: "securepassword",
});
// Social (Google, GitHub)
await auth.signIn.social({
provider: "google", // or "github"
callbackURL: "/dashboard",
});
```
### Sign Out
```typescript
await auth.signOut();
```
### Get Session
```typescript
// Async (Node.js, server components)
const session = await auth.getSession();
// React hook (client components)
const session = auth.useSession();
// Returns: { data: Session | null, isPending: boolean }
```
## Session Data Structure
```typescript
interface Session {
user: {
id: string;
name: string | null;
email: string;
image: string | null;
emailVerified: boolean;
createdAt: Date;
updatedAt: Date;
};
session: {
id: string;
expiresAt: Date;
token: string;
createdAt: Date;
updatedAt: Date;
userId: string;
};
}
```
## Error Handling
```typescript
const { error } = await auth.signIn.email({ email, password });
if (error) {
switch (error.code) {
case "INVALID_EMAIL_OR_PASSWORD":
showError("Invalid email or password");
break;
case "EMAIL_NOT_VERIFIED":
showError("Please verify your email");
break;
case "USER_NOT_FOUND":
showError("User not found");
break;
case "TOO_MANY_REQUESTS":
showError("Too many attempts. Please wait.");
break;
default:
showError("Authentication failed");
}
}
```
## Building Auth Pages
### Use AuthView (Recommended for React Apps)
For authentication pages, use the pre-built `AuthView` component instead of building custom forms.
**What AuthView provides:**
- Sign-in, sign-up, password reset, magic link pages
- Social providers (Google, GitHub) - requires TWO configurations: enable in Neon Console AND add `social` prop to NeonAuthUIProvider
- Form validation, error handling, loading states
- Consistent styling via CSS variables
**Setup (Next.js App Router):**
1. **Import CSS** (in `app/layout.tsx` or `app/globals.css`):
```tsx
import "@neondatabase/auth/ui/css";
```
2. **Wrap app with provider** (create `app/auth-provider.tsx`):
```tsx
"use client";
import { NeonAuthUIProvider } from "@neondatabase/auth/react/ui";
import { authClient } from "@/lib/auth/client";
import { useRouter } from "next/navigation";
import Link from "next/link";
export function AuthProvider({ children }: { children: React.ReactNode }) {
const router = useRouter();
return (
<NeonAuthUIProvider
authClient={authClient}
navigate={router.push}
replace={router.replace}
onSessionChange={() => router.refresh()}
Link={Link}
>
{children}
</NeonAuthUIProvider>
);
}
```
3. **Create auth page** (`app/auth/[path]/page.tsx`):
```tsx
import { AuthView } from "@neondatabase/auth/react/ui";
import { authViewPaths } from "@neondatabase/auth/react/ui/server";
export function generateStaticParams() {
return Object.values(authViewPaths).map((path) => ({ path }));
}
export default async function AuthPage({
params,
}: {
params: Promise<{ path: string }>;
}) {
const { path } = await params;
return <AuthView pathname={path} />;
}
```
**Result:** You now have `/auth/sign-in`, `/auth/sign-up`, `/auth/forgot-password`, etc.
**Available paths:** `"sign-in"`, `"sign-up"`, `"forgot-password"`, `"reset-password"`, `"magic-link"`, `"two-factor"`, `"callback"`, `"sign-out"`
### When to Use Low-Level Methods Instead
Use `authClient.signIn.email()`, `authClient.signUp.email()` directly if:
- **Node.js backend** - No React, server-side auth only
- **Custom design system** - Your design team provides form components
- **Mobile/CLI apps** - Non-web frontends
- **Headless auth** - Testing or non-standard flows
For standard React web apps, **use AuthView**.
### Common Anti-Pattern
```tsx
// ❌ Don't build custom forms unless you have specific requirements
function CustomSignInPage() {
const [email, setEmail] = useState("");
const [password, setPassword] = useState("");
const [error, setError] = useState("");
const [loading, setLoading] = useState(false);
const handleSubmit = async (e) => {
e.preventDefault();
setLoading(true);
const { error } = await authClient.signIn.email({ email, password });
if (error) setError(error.message);
setLoading(false);
};
// ... 50+ more lines of form JSX, validation, error display
}
// ✅ Use AuthView instead - one component handles everything
<AuthView pathname="sign-in" />;
```
## Styling
Neon Auth UI **automatically inherits your app's existing theme**. If you have CSS variables like `--primary`, `--background`, etc. defined (from Tailwind, shadcn/ui, or custom CSS), auth components use them with no configuration.
**Key features:**
- **Automatic inheritance**: Uses your existing `--primary`, `--background`, etc.
- **No conflicts**: Auth styles are in `@layer neon-auth`, so your styles always win
- **Import order doesn't matter**: CSS layers handle priority automatically
### Integration with shadcn/ui
If you use shadcn/ui or similar libraries that define `--primary`, `--background`, etc., Neon Auth will automatically inherit those colors. No additional configuration needed.
### Use Existing CSS Variables
When creating custom components, use CSS variables for consistency:
| Variable | Purpose |
| ----------------------------------- | ----------------------- |
| `--background`, `--foreground` | Page background/text |
| `--card`, `--card-foreground` | Card surfaces |
| `--primary`, `--primary-foreground` | Primary buttons/actions |
| `--muted`, `--muted-foreground` | Muted/subtle elements |
| `--border`, `--ring` | Borders and focus rings |
| `--radius` | Border radius |
### Auth-Specific Customization
To customize auth components differently from your main app, use `--neon-*` prefix:
```css
:root {
--primary: oklch(0.55 0.25 250); /* Your app's blue */
--neon-primary: oklch(0.55 0.18 145); /* Auth uses green */
}
```
@@ -0,0 +1,225 @@
# Neon Auth - Common Mistakes
Reference guide for common mistakes when using `@neondatabase/auth` or `@neondatabase/neon-js`.
## Import Mistakes
### BetterAuthReactAdapter Subpath Requirement
`BetterAuthReactAdapter` is **NOT** exported from the main package entry. You must import it from the subpath.
**Wrong:**
```typescript
// These will NOT work
import { BetterAuthReactAdapter } from "@neondatabase/neon-js";
import { BetterAuthReactAdapter } from "@neondatabase/auth";
```
**Correct:**
```typescript
// For @neondatabase/neon-js
import { BetterAuthReactAdapter } from "@neondatabase/neon-js/auth/react/adapters";
// For @neondatabase/auth
import { BetterAuthReactAdapter } from "@neondatabase/auth/react/adapters";
```
**Why:** The React adapter has React-specific dependencies and is tree-shaken out of the main bundle. Using subpath exports keeps the main bundle smaller for non-React environments.
### Adapter Factory Functions
All adapters are **factory functions** that must be called with `()`.
**Wrong:**
```typescript
const client = createClient({
auth: {
adapter: BetterAuthReactAdapter, // Missing ()
url: process.env.NEON_AUTH_URL!,
},
dataApi: { url: process.env.NEON_DATA_API_URL! },
});
```
**Correct:**
```typescript
const client = createClient({
auth: {
adapter: BetterAuthReactAdapter(), // Called as function
url: process.env.NEON_AUTH_URL!,
},
dataApi: { url: process.env.NEON_DATA_API_URL! },
});
```
This applies to all adapters:
- `BetterAuthReactAdapter()`
- `BetterAuthVanillaAdapter()`
- `SupabaseAuthAdapter()`
---
## CSS Import Mistakes
Auth UI components require CSS. Choose **ONE** method based on your project.
### With Tailwind v4
```css
/* In app/globals.css */
@import "tailwindcss";
@import "@neondatabase/neon-js/ui/tailwind";
/* Or: @import '@neondatabase/auth/ui/tailwind'; */
```
### Without Tailwind
```typescript
// In app/layout.tsx
import "@neondatabase/neon-js/ui/css";
// Or: import "@neondatabase/auth/ui/css";
```
### Never Import Both
**Wrong:**
```css
/* Causes ~94KB of duplicate styles */
@import "@neondatabase/neon-js/ui/css";
@import "@neondatabase/neon-js/ui/tailwind";
```
**Why:** The `ui/css` import includes pre-built CSS (~47KB). The `ui/tailwind` import provides Tailwind tokens (~2KB) that generate similar styles. Using both doubles your CSS bundle.
---
## Configuration Mistakes
### Wrong createAuthClient Signature
The `createAuthClient` function takes the URL as the first argument, not as a property in an options object.
**Wrong:**
```typescript
// This will NOT work
createAuthClient({ baseURL: url });
createAuthClient({ url: myUrl });
```
**Correct:**
```typescript
// Vanilla client - URL as first arg
createAuthClient(url);
// With adapter - URL as first arg, options as second
createAuthClient(url, { adapter: BetterAuthReactAdapter() });
// Next.js client - no arguments (uses env vars automatically)
import { createAuthClient } from "@neondatabase/auth/next";
const authClient = createAuthClient();
```
### Missing Environment Variables
**Required for Next.js:**
```bash
# .env.local
NEON_AUTH_BASE_URL=https://ep-xxx.neonauth.c-2.us-east-2.aws.neon.build/dbname/auth
NEXT_PUBLIC_NEON_AUTH_URL=https://ep-xxx.neonauth.c-2.us-east-2.aws.neon.build/dbname/auth
# For neon-js (auth + data)
NEON_DATA_API_URL=https://ep-xxx.apirest.c-2.us-east-2.aws.neon.build/dbname/rest/v1
```
**Required for Vite/React SPA:**
```bash
# .env
VITE_NEON_AUTH_URL=https://ep-xxx.neonauth.c-2.us-east-2.aws.neon.build/dbname/auth
VITE_NEON_DATA_API_URL=https://ep-xxx.apirest.c-2.us-east-2.aws.neon.build/dbname/rest/v1
```
**Important:**
- `NEON_AUTH_BASE_URL` - Server-side auth
- `NEXT_PUBLIC_*` prefix - Required for client-side access in Next.js
- `VITE_*` prefix - Required for client-side access in Vite
- Restart dev server after adding env vars
---
## Usage Mistakes
### Missing "use client" Directive
Client components using `useSession()` need the `"use client"` directive.
**Wrong:**
```typescript
// Missing directive - will cause hydration errors
import { authClient } from "@/lib/auth/client";
function AuthStatus() {
const session = authClient.useSession();
// ...
}
```
**Correct:**
```typescript
"use client";
import { authClient } from "@/lib/auth/client";
function AuthStatus() {
const session = authClient.useSession();
// ...
}
```
### Wrong API for Adapter
Each adapter has its own API style. Don't mix them.
**Wrong - BetterAuth API with SupabaseAuthAdapter:**
```typescript
const client = createClient({
auth: { adapter: SupabaseAuthAdapter(), url },
dataApi: { url },
});
// This won't work with SupabaseAuthAdapter
await client.auth.signIn.email({ email, password });
```
**Correct - Supabase API with SupabaseAuthAdapter:**
```typescript
const client = createClient({
auth: { adapter: SupabaseAuthAdapter(), url },
dataApi: { url },
});
// Use Supabase-style methods
await client.auth.signInWithPassword({ email, password });
```
**API Reference by Adapter:**
| Adapter | Sign In | Sign Up | Get Session |
| ------------------------ | ----------------------------------------- | ----------------------------------- | ------------------------------- |
| BetterAuthVanillaAdapter | `signIn.email({ email, password })` | `signUp.email({ email, password })` | `getSession()` |
| BetterAuthReactAdapter | `signIn.email({ email, password })` | `signUp.email({ email, password })` | `useSession()` / `getSession()` |
| SupabaseAuthAdapter | `signInWithPassword({ email, password })` | `signUp({ email, password })` | `getSession()` |
@@ -0,0 +1,110 @@
# Neon Auth Setup - Next.js App Router
Complete setup instructions for Neon Auth in Next.js App Router applications.
---
## 1. Install Package
```bash
npm install @neondatabase/auth
# Or: npm install @neondatabase/neon-js
```
## 2. Environment Variables
Create or update `.env.local`:
```bash
NEON_AUTH_BASE_URL=https://ep-xxx.neonauth.c-2.us-east-2.aws.neon.build/dbname/auth
NEXT_PUBLIC_NEON_AUTH_URL=https://ep-xxx.neonauth.c-2.us-east-2.aws.neon.build/dbname/auth
```
**Important:** Both variables are needed:
- `NEON_AUTH_BASE_URL` - Used by server-side API routes
- `NEXT_PUBLIC_NEON_AUTH_URL` - Used by client-side components (prefixed with NEXT*PUBLIC*)
**Where to find your Auth URL:**
1. Go to your Neon project dashboard
2. Navigate to the "Auth" tab
3. Copy the Auth URL
## 3. API Route Handler
Create `app/api/auth/[...path]/route.ts`:
```typescript
import { authApiHandler } from "@neondatabase/auth/next";
// Or: import { authApiHandler } from "@neondatabase/neon-js/auth/next";
export const { GET, POST } = authApiHandler();
```
This creates endpoints for:
- `/api/auth/sign-in` - Sign in
- `/api/auth/sign-up` - Sign up
- `/api/auth/sign-out` - Sign out
- `/api/auth/session` - Get session
- And other auth-related endpoints
## 4. Auth Client Configuration
Create `lib/auth/client.ts`:
```typescript
import { createAuthClient } from "@neondatabase/auth/next";
// Or: import { createAuthClient } from "@neondatabase/neon-js/auth/next";
export const authClient = createAuthClient();
```
## 5. Use in Components
```typescript
"use client";
import { authClient } from "@/lib/auth/client";
function AuthStatus() {
const session = authClient.useSession();
if (session.isPending) return <div>Loading...</div>;
if (!session.data) return <SignInButton />;
return (
<div>
<p>Hello, {session.data.user.name}</p>
<button onClick={() => authClient.signOut()}>Sign Out</button>
</div>
);
}
function SignInButton() {
return (
<button onClick={() => authClient.signIn.email({
email: "user@example.com",
password: "password"
})}>
Sign In
</button>
);
}
```
## 6. UI Provider Setup (Optional)
For pre-built UI components (AuthView, UserButton, etc.), see `ui-components.md`.
---
## Package Selection
| Need | Package | Bundle Size |
| ----------------------- | ----------------------- | --------------- |
| Auth only | `@neondatabase/auth` | Smaller (~50KB) |
| Auth + Database queries | `@neondatabase/neon-js` | Full (~150KB) |
**Recommendation:** Use `@neondatabase/auth` if you only need authentication. Use `@neondatabase/neon-js` if you also need PostgREST-style database queries.
@@ -0,0 +1,248 @@
# Neon Auth Setup - React SPA (Vite)
Complete setup instructions for Neon Auth in React Single Page Applications (Vite, Create React App, etc.).
---
## 1. Install Package
```bash
npm install @neondatabase/auth
# Or: npm install @neondatabase/neon-js
npm install react-router-dom # Required for UI components
```
## 2. Environment Variables
Create or update `.env`:
**For Vite:**
```bash
VITE_NEON_AUTH_URL=https://ep-xxx.neonauth.c-2.us-east-2.aws.neon.build/dbname/auth
```
**For Create React App:**
```bash
REACT_APP_NEON_AUTH_URL=https://ep-xxx.neonauth.c-2.us-east-2.aws.neon.build/dbname/auth
```
**Where to find your Auth URL:**
1. Go to your Neon project dashboard
2. Navigate to the "Auth" tab
3. Copy the Auth URL
## 3. Auth Client Configuration
Create `src/lib/auth-client.ts`:
**For `@neondatabase/auth`:**
```typescript
import { createAuthClient } from "@neondatabase/auth";
import { BetterAuthReactAdapter } from "@neondatabase/auth/react/adapters";
export const authClient = createAuthClient(import.meta.env.VITE_NEON_AUTH_URL, {
adapter: BetterAuthReactAdapter(),
});
```
**For `@neondatabase/neon-js`:**
```typescript
import { createClient } from "@neondatabase/neon-js";
import { BetterAuthReactAdapter } from "@neondatabase/neon-js/auth/react/adapters";
export const client = createClient({
auth: {
adapter: BetterAuthReactAdapter(),
url: import.meta.env.VITE_NEON_AUTH_URL,
},
dataApi: {
url: import.meta.env.VITE_NEON_DATA_API_URL,
},
});
export const authClient = client.auth;
```
**Critical:**
- `BetterAuthReactAdapter` must be imported from the `/react/adapters` subpath
- The adapter must be called as a function: `BetterAuthReactAdapter()`
## 4. Use in Components
```typescript
import { authClient } from "./lib/auth-client";
function App() {
const session = authClient.useSession();
if (session.isPending) return <div>Loading...</div>;
if (!session.data) return <LoginForm />;
return <Dashboard user={session.data.user} />;
}
```
---
## 5. UI Provider Setup (Optional)
Skip this section if you're building custom auth forms. Use this if you want pre-built UI components.
### 5a. Import CSS
**CRITICAL:** Choose ONE import method. Never import both - it causes duplicate styles.
**Check if the project uses Tailwind CSS** by looking for:
- `tailwind.config.js` or `tailwind.config.ts` in the project root
- `@import 'tailwindcss'` or `@tailwind` directives in CSS files
- `tailwindcss` in package.json dependencies
**If NOT using Tailwind** - Add to `src/main.tsx` or entry point:
```typescript
import "@neondatabase/auth/ui/css";
```
**If using Tailwind CSS v4** - Add to main CSS file (e.g., index.css):
```css
@import "tailwindcss";
@import "@neondatabase/auth/ui/tailwind";
```
### 5b. Update main.tsx with BrowserRouter
```tsx
import { createRoot } from "react-dom/client";
import { BrowserRouter } from "react-router-dom";
import "@neondatabase/auth/ui/css"; // if not using Tailwind
import App from "./App";
import { Providers } from "./providers";
createRoot(document.getElementById("root")!).render(
<BrowserRouter>
<Providers>
<App />
</Providers>
</BrowserRouter>,
);
```
### 5c. Create Auth Provider
Create `src/providers.tsx`:
```tsx
import { NeonAuthUIProvider } from "@neondatabase/auth/react/ui";
import { useNavigate, Link as RouterLink } from "react-router-dom";
import { authClient } from "./lib/auth-client";
import type { ReactNode } from "react";
// Adapter for react-router-dom Link
function Link({
href,
...props
}: { href: string } & React.AnchorHTMLAttributes<HTMLAnchorElement>) {
return <RouterLink to={href} {...props} />;
}
export function Providers({ children }: { children: ReactNode }) {
const navigate = useNavigate();
return (
<NeonAuthUIProvider
authClient={authClient}
navigate={(path) => navigate(path)}
replace={(path) => navigate(path, { replace: true })}
onSessionChange={() => {
// Optional: refresh data or invalidate cache
}}
Link={Link}
social={{
providers: ["google", "github"],
}}
>
{children}
</NeonAuthUIProvider>
);
}
```
**Provider props explained:**
- `navigate`: Function to navigate to a new route
- `replace`: Function to replace current route (for redirects)
- `onSessionChange`: Callback when auth state changes (useful for cache invalidation)
- `Link`: Adapter component for react-router-dom's Link
- `social`: Show Google and GitHub sign-in buttons (both enabled by default in Neon)
### 5d. Add Routes to App.tsx
```tsx
import { Routes, Route, useParams } from "react-router-dom";
import {
AuthView,
UserButton,
SignedIn,
SignedOut,
} from "@neondatabase/auth/react/ui";
// Auth page - handles /auth/sign-in, /auth/sign-up, etc.
function AuthPage() {
const { pathname } = useParams();
return (
<div className="flex min-h-screen items-center justify-center">
<AuthView pathname={pathname} />
</div>
);
}
// Simple navbar example
function Navbar() {
return (
<nav className="flex items-center justify-between p-4 border-b">
<a href="/">My App</a>
<div className="flex items-center gap-4">
<SignedOut>
<a href="/auth/sign-in">Sign In</a>
</SignedOut>
<SignedIn>
<UserButton />
</SignedIn>
</div>
</nav>
);
}
function HomePage() {
return <div>Welcome to My App!</div>;
}
export default function App() {
return (
<>
<Navbar />
<Routes>
<Route path="/" element={<HomePage />} />
<Route path="/auth/:pathname" element={<AuthPage />} />
</Routes>
</>
);
}
```
**Auth routes created:**
- `/auth/sign-in` - Sign in page
- `/auth/sign-up` - Sign up page
- `/auth/forgot-password` - Password reset request
- `/auth/reset-password` - Set new password
- `/auth/sign-out` - Sign out
- `/auth/callback` - OAuth callback (internal)
@@ -0,0 +1,215 @@
# Neon Auth - UI Components Reference
Pre-built UI components for authentication flows.
## Available Components
- `AuthView` - Complete auth pages (sign-in, sign-up, forgot-password, etc.) - **use this first**
- `SignedIn` / `SignedOut` - Conditional rendering based on auth state
- `UserButton` - User avatar with dropdown menu
- `NeonAuthUIProvider` - Required wrapper for UI components
## CSS Import
**CRITICAL:** Choose ONE import method. Never import both.
**Without Tailwind:**
```typescript
// In app/layout.tsx or entry point
import "@neondatabase/auth/ui/css";
```
**With Tailwind v4:**
```css
/* In app/globals.css */
@import "tailwindcss";
@import "@neondatabase/auth/ui/tailwind";
```
## NeonAuthUIProvider Setup
### Next.js App Router
```tsx
"use client";
import { NeonAuthUIProvider } from "@neondatabase/auth/react/ui";
import { authClient } from "@/lib/auth/client";
import { useRouter } from "next/navigation";
import Link from "next/link";
export function AuthProvider({ children }: { children: React.ReactNode }) {
const router = useRouter();
return (
<NeonAuthUIProvider
authClient={authClient}
navigate={router.push}
replace={router.replace}
onSessionChange={() => router.refresh()}
Link={Link}
social={{
providers: ["google", "github"],
}}
>
{children}
</NeonAuthUIProvider>
);
}
```
### React SPA with react-router-dom
```tsx
import { NeonAuthUIProvider } from "@neondatabase/auth/react/ui";
import { useNavigate, Link as RouterLink } from "react-router-dom";
import { authClient } from "./lib/auth-client";
function Link({
href,
...props
}: { href: string } & React.AnchorHTMLAttributes<HTMLAnchorElement>) {
return <RouterLink to={href} {...props} />;
}
export function Providers({ children }: { children: React.ReactNode }) {
const navigate = useNavigate();
return (
<NeonAuthUIProvider
authClient={authClient}
navigate={(path) => navigate(path)}
replace={(path) => navigate(path, { replace: true })}
onSessionChange={() => {}}
Link={Link}
social={{
providers: ["google", "github"],
}}
>
{children}
</NeonAuthUIProvider>
);
}
```
## AuthView Component
Renders complete authentication pages.
### Next.js App Router
Create `app/auth/[path]/page.tsx`:
```tsx
import { AuthView } from "@neondatabase/auth/react/ui";
import { authViewPaths } from "@neondatabase/auth/react/ui/server";
export function generateStaticParams() {
return Object.values(authViewPaths).map((path) => ({ path }));
}
export default async function AuthPage({
params,
}: {
params: Promise<{ path: string }>;
}) {
const { path } = await params;
return <AuthView pathname={path} />;
}
```
### React SPA
```tsx
import { Routes, Route, useParams } from "react-router-dom";
import { AuthView } from "@neondatabase/auth/react/ui";
function AuthPage() {
const { pathname } = useParams();
return (
<div className="flex min-h-screen items-center justify-center">
<AuthView pathname={pathname} />
</div>
);
}
export default function App() {
return (
<Routes>
<Route path="/" element={<HomePage />} />
<Route path="/auth/:pathname" element={<AuthPage />} />
</Routes>
);
}
```
### Available Auth Paths
| Path | Purpose |
| ----------------- | ------------------------- |
| `sign-in` | Sign in page |
| `sign-up` | Sign up page |
| `forgot-password` | Password reset request |
| `reset-password` | Set new password |
| `magic-link` | Magic link sign in |
| `two-factor` | Two-factor authentication |
| `callback` | OAuth callback (internal) |
| `sign-out` | Sign out |
## SignedIn / SignedOut Components
Conditional rendering based on authentication state.
```tsx
import { SignedIn, SignedOut, UserButton } from "@neondatabase/auth/react/ui";
function Navbar() {
return (
<nav>
<SignedOut>
<a href="/auth/sign-in">Sign In</a>
<a href="/auth/sign-up">Sign Up</a>
</SignedOut>
<SignedIn>
<UserButton />
</SignedIn>
</nav>
);
}
```
## UserButton Component
Displays user avatar with dropdown menu for account management.
```tsx
import { UserButton } from "@neondatabase/auth/react/ui";
function Header() {
return (
<header>
<h1>My App</h1>
<UserButton />
</header>
);
}
```
## Social Login Configuration
**Important:** Social providers require TWO configurations:
1. **Enable in Neon Console** - Go to your project's Auth settings
2. **Add to NeonAuthUIProvider** - Pass `social` prop
```tsx
<NeonAuthUIProvider
authClient={authClient}
// ... other props
social={{
providers: ['google', 'github']
}}
>
```
Without both configurations, social login buttons won't appear.
@@ -0,0 +1,164 @@
# Neon CLI
The Neon CLI is a command-line interface for managing Neon Serverless Postgres directly from your terminal. It provides the same capabilities as the Neon Platform API and is ideal for scripting, CI/CD pipelines, and developers who prefer terminal workflows.
## Installation
**macOS (Homebrew):**
```bash
brew install neonctl
```
**npm (cross-platform):**
```bash
npm install -g neonctl
```
**Direct download:**
```bash
curl -fsSL https://neon.tech/install.sh | bash
```
## Authentication
Authenticate with your Neon account:
```bash
neonctl auth
```
This opens a browser for OAuth authentication and stores credentials locally.
For CI/CD or non-interactive environments, use an API key:
```bash
export NEON_API_KEY=your-api-key
```
Get your API key from: https://console.neon.tech/app/settings/api-keys
## Common Commands
### Project Management
```bash
# List all projects
neonctl projects list
# Create a new project
neonctl projects create --name my-project
# Get project details
neonctl projects get <project-id>
# Delete a project
neonctl projects delete <project-id>
```
### Branch Operations
```bash
# List branches
neonctl branches list --project-id <project-id>
# Create a branch
neonctl branches create --project-id <project-id> --name dev
# Delete a branch
neonctl branches delete <branch-id> --project-id <project-id>
```
### Connection Strings
```bash
# Get connection string
neonctl connection-string --project-id <project-id>
# Get connection string for specific branch
neonctl connection-string --project-id <project-id> --branch-id <branch-id>
# Get pooled connection string
neonctl connection-string --project-id <project-id> --pooled
```
### SQL Execution
```bash
# Run SQL query
neonctl sql "SELECT * FROM users LIMIT 10" --project-id <project-id>
# Run SQL from file
neonctl sql --file schema.sql --project-id <project-id>
```
### Database Management
```bash
# List databases
neonctl databases list --project-id <project-id> --branch-id <branch-id>
# Create database
neonctl databases create --project-id <project-id> --name mydb
# List roles
neonctl roles list --project-id <project-id> --branch-id <branch-id>
```
## Output Formats
The CLI supports multiple output formats:
```bash
# JSON output (default for scripting)
neonctl projects list --output json
# Table output (human-readable)
neonctl projects list --output table
# YAML output
neonctl projects list --output yaml
```
## CI/CD Integration
Example GitHub Actions workflow:
```yaml
- name: Create preview branch
env:
NEON_API_KEY: ${{ secrets.NEON_API_KEY }}
run: |
neonctl branches create \
--project-id ${{ vars.NEON_PROJECT_ID }} \
--name preview-${{ github.event.pull_request.number }}
```
## CLI vs MCP Server vs SDKs
| Tool | Best For |
| -------------- | ------------------------------------------------- |
| Neon CLI | Terminal workflows, scripts, CI/CD pipelines |
| MCP Server | AI-assisted development with Claude, Cursor, etc. |
| TypeScript SDK | Programmatic access in Node.js/TypeScript apps |
| Python SDK | Programmatic access in Python applications |
| REST API | Direct HTTP integration in any language |
## Documentation Resources
| Topic | URL |
| -------------- | ------------------------------------------------------ |
| CLI Reference | https://neon.tech/docs/reference/neon-cli |
| CLI Install | https://neon.tech/docs/reference/cli-install |
| CLI Auth | https://neon.tech/docs/reference/cli-auth |
| CLI Projects | https://neon.tech/docs/reference/cli-projects |
| CLI Branches | https://neon.tech/docs/reference/cli-branches |
| CLI Connection | https://neon.tech/docs/reference/cli-connection-string |
Fetch CLI documentation:
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/reference/neon-cli
```
@@ -0,0 +1,245 @@
# Neon and Drizzle Integration
Integration patterns, configurations, and optimizations for using **Drizzle ORM** with **Neon** Postgres.
For official documentation:
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/guides/drizzle
```
## Choosing the Right Driver
Drizzle ORM works with multiple Postgres drivers. See `connection-methods.md` for the full decision tree.
| Platform | TCP Support | Pooling | Recommended Driver |
| ----------------------- | ----------- | ------------------- | -------------------------- |
| Vercel (Fluid) | Yes | `@vercel/functions` | `pg` (node-postgres) |
| Cloudflare (Hyperdrive) | Yes | Hyperdrive | `pg` (node-postgres) |
| Cloudflare Workers | No | No | `@neondatabase/serverless` |
| Netlify Functions | No | No | `@neondatabase/serverless` |
| Deno Deploy | No | No | `@neondatabase/serverless` |
| Railway / Render | Yes | Built-in | `pg` (node-postgres) |
## Connection Setup
### 1. TCP with node-postgres (Long-Running Servers)
Best for Railway, Render, traditional VPS.
```bash
npm install drizzle-orm pg
npm install -D drizzle-kit @types/pg dotenv
```
```typescript
// src/db.ts
import { drizzle } from "drizzle-orm/node-postgres";
import { Pool } from "pg";
const pool = new Pool({ connectionString: process.env.DATABASE_URL });
export const db = drizzle({ client: pool });
```
### 2. Vercel Fluid Compute with Connection Pooling
```bash
npm install drizzle-orm pg @vercel/functions
npm install -D drizzle-kit @types/pg
```
```typescript
// src/db.ts
import { attachDatabasePool } from "@vercel/functions";
import { drizzle } from "drizzle-orm/node-postgres";
import { Pool } from "pg";
import * as schema from "./schema";
const pool = new Pool({ connectionString: process.env.DATABASE_URL });
attachDatabasePool(pool);
export const db = drizzle({ client: pool, schema });
```
### 3. HTTP Adapter (Edge Without TCP)
For Cloudflare Workers, Netlify Edge, Deno Deploy. Does NOT support interactive transactions.
```bash
npm install drizzle-orm @neondatabase/serverless
npm install -D drizzle-kit dotenv
```
```typescript
// src/db.ts
import { drizzle } from "drizzle-orm/neon-http";
import { neon } from "@neondatabase/serverless";
const sql = neon(process.env.DATABASE_URL!);
export const db = drizzle(sql);
```
### 4. WebSocket Adapter (Edge with Transactions)
```bash
npm install drizzle-orm @neondatabase/serverless ws
npm install -D drizzle-kit dotenv @types/ws
```
```typescript
// src/db.ts
import { drizzle } from "drizzle-orm/neon-serverless";
import { Pool, neonConfig } from "@neondatabase/serverless";
import ws from "ws";
neonConfig.webSocketConstructor = ws; // Required for Node.js < v22
const pool = new Pool({ connectionString: process.env.DATABASE_URL });
export const db = drizzle(pool);
```
## Drizzle Config
```typescript
// drizzle.config.ts
import { config } from "dotenv";
import { defineConfig } from "drizzle-kit";
config({ path: ".env.local" });
export default defineConfig({
schema: "./src/schema.ts",
out: "./drizzle",
dialect: "postgresql",
dbCredentials: {
url: process.env.DATABASE_URL!,
},
});
```
## Migrations
```bash
# Generate migrations
npx drizzle-kit generate
# Apply migrations
npx drizzle-kit migrate
```
## Schema Definition
```typescript
// src/schema.ts
import { pgTable, serial, text, integer, timestamp } from "drizzle-orm/pg-core";
export const usersTable = pgTable("users", {
id: serial("id").primaryKey(),
name: text("name").notNull(),
email: text("email").notNull().unique(),
role: text("role").default("user").notNull(),
createdAt: timestamp("created_at").defaultNow().notNull(),
});
export type User = typeof usersTable.$inferSelect;
export type NewUser = typeof usersTable.$inferInsert;
export const postsTable = pgTable("posts", {
id: serial("id").primaryKey(),
title: text("title").notNull(),
content: text("content").notNull(),
userId: integer("user_id")
.notNull()
.references(() => usersTable.id, { onDelete: "cascade" }),
createdAt: timestamp("created_at").defaultNow().notNull(),
});
export type Post = typeof postsTable.$inferSelect;
export type NewPost = typeof postsTable.$inferInsert;
```
## Query Patterns
### Batch Inserts
```typescript
export async function batchInsertUsers(users: NewUser[]) {
return db.insert(usersTable).values(users).returning();
}
```
### Prepared Statements
```typescript
import { sql } from "drizzle-orm";
export const getUsersByRolePrepared = db
.select()
.from(usersTable)
.where(sql`${usersTable.role} = $1`)
.prepare("get_users_by_role");
// Usage: getUsersByRolePrepared.execute(['admin'])
```
### Transactions
```typescript
export async function createUserWithPosts(user: NewUser, posts: NewPost[]) {
return await db.transaction(async (tx) => {
const [newUser] = await tx.insert(usersTable).values(user).returning();
if (posts.length > 0) {
await tx.insert(postsTable).values(
posts.map((post) => ({
...post,
userId: newUser.id,
})),
);
}
return newUser;
});
}
```
## Working with Neon Branches
```typescript
import { drizzle } from "drizzle-orm/neon-http";
import { neon } from "@neondatabase/serverless";
const getBranchUrl = () => {
const env = process.env.NODE_ENV;
if (env === "development") return process.env.DEV_DATABASE_URL;
if (env === "test") return process.env.TEST_DATABASE_URL;
return process.env.DATABASE_URL;
};
const sql = neon(getBranchUrl()!);
export const db = drizzle({ client: sql });
```
## Error Handling
```typescript
export async function safeNeonOperation<T>(
operation: () => Promise<T>,
): Promise<T> {
try {
return await operation();
} catch (error: any) {
if (error.message?.includes("connection pool timeout")) {
console.error("Neon connection pool timeout");
}
throw error;
}
}
```
## Best Practices
1. **Connection Management** - See `connection-methods.md` for platform-specific guidance
2. **Neon Features** - Utilize branching for development/testing (see `features.md`)
3. **Query Optimization** - Batch operations, use prepared statements
4. **Schema Design** - Leverage Postgres-specific features, use appropriate indexes
@@ -0,0 +1,218 @@
# Neon JS SDK
The `@neondatabase/neon-js` SDK provides a unified client for Neon Auth and Data API. It combines authentication handling with PostgREST-compatible database queries.
**Auth only?** Use `neon-auth.md` instead for smaller bundle size.
For official documentation:
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/reference/javascript-sdk
```
## Package Selection
| Use Case | Package | Notes |
| --------------- | ---------------------------- | ------------------- |
| Auth + Data API | `@neondatabase/neon-js` | Full SDK |
| Auth only | `@neondatabase/auth` | Smaller bundle |
| Data API only | `@neondatabase/postgrest-js` | Bring your own auth |
## Installation
```bash
npm install @neondatabase/neon-js
```
## Quick Setup Patterns
### Next.js (Most Common)
**1. API Route Handler:**
```typescript
// app/api/auth/[...path]/route.ts
import { authApiHandler } from "@neondatabase/neon-js/auth/next";
export const { GET, POST } = authApiHandler();
```
**2. Auth Client:**
```typescript
// lib/auth/client.ts
import { createAuthClient } from "@neondatabase/neon-js/auth/next";
export const authClient = createAuthClient();
```
**3. Database Client:**
```typescript
// lib/db/client.ts
import { createClient } from "@neondatabase/neon-js";
import type { Database } from "./database.types";
export const dbClient = createClient<Database>({
auth: { url: process.env.NEXT_PUBLIC_NEON_AUTH_URL! },
dataApi: { url: process.env.NEON_DATA_API_URL! },
});
```
### React SPA
```typescript
import { createClient } from "@neondatabase/neon-js";
import { BetterAuthReactAdapter } from "@neondatabase/neon-js/auth/react/adapters";
const client = createClient<Database>({
auth: {
adapter: BetterAuthReactAdapter(),
url: import.meta.env.VITE_NEON_AUTH_URL,
},
dataApi: { url: import.meta.env.VITE_NEON_DATA_API_URL },
});
```
### Node.js Backend
```typescript
import { createClient } from "@neondatabase/neon-js";
const client = createClient<Database>({
auth: { url: process.env.NEON_AUTH_URL! },
dataApi: { url: process.env.NEON_DATA_API_URL! },
});
```
## Environment Variables
```bash
# Next.js (.env.local)
NEON_AUTH_BASE_URL=https://ep-xxx.neonauth.c-2.us-east-2.aws.neon.build/dbname/auth
NEXT_PUBLIC_NEON_AUTH_URL=https://ep-xxx.neonauth.c-2.us-east-2.aws.neon.build/dbname/auth
NEON_DATA_API_URL=https://ep-xxx.apirest.c-2.us-east-2.aws.neon.build/dbname/rest/v1
# Vite/React (.env)
VITE_NEON_AUTH_URL=https://ep-xxx.neonauth.c-2.us-east-2.aws.neon.build/dbname/auth
VITE_NEON_DATA_API_URL=https://ep-xxx.apirest.c-2.us-east-2.aws.neon.build/dbname/rest/v1
```
## Database Queries
All query methods follow PostgREST syntax (same as Supabase):
```typescript
// Select with filters
const { data } = await client
.from("items")
.select("id, name, status")
.eq("status", "active")
.order("created_at", { ascending: false })
.limit(10);
// Insert
const { data, error } = await client
.from("items")
.insert({ name: "New Item", status: "pending" })
.select()
.single();
// Update
await client.from("items").update({ status: "completed" }).eq("id", 1);
// Delete
await client.from("items").delete().eq("id", 1);
```
For complete Data API query reference, see `neon-js/data-api.md`.
## Auth Methods
### BetterAuth API (Default)
```typescript
// Sign in/up
await client.auth.signIn.email({ email, password });
await client.auth.signUp.email({ email, password, name });
await client.auth.signOut();
// Get session
const session = await client.auth.getSession();
// Social sign-in
await client.auth.signIn.social({
provider: "google",
callbackURL: "/dashboard",
});
```
### Supabase-Compatible API
```typescript
import { createClient, SupabaseAuthAdapter } from "@neondatabase/neon-js";
const client = createClient({
auth: { adapter: SupabaseAuthAdapter(), url },
dataApi: { url },
});
await client.auth.signInWithPassword({ email, password });
await client.auth.signUp({ email, password });
const {
data: { session },
} = await client.auth.getSession();
```
## Sub-Resources
| Topic | Resource |
| ---------------- | ---------------------------- |
| Data API queries | `neon-js/data-api.md` |
| Common mistakes | `neon-js/common-mistakes.md` |
## Key Imports
```typescript
// Main client
import {
createClient,
SupabaseAuthAdapter,
BetterAuthVanillaAdapter,
} from "@neondatabase/neon-js";
// Next.js integration
import {
authApiHandler,
createAuthClient,
} from "@neondatabase/neon-js/auth/next";
// React adapter (NOT from main entry - must use subpath)
import { BetterAuthReactAdapter } from "@neondatabase/neon-js/auth/react/adapters";
// UI components
import {
NeonAuthUIProvider,
AuthView,
SignInForm,
} from "@neondatabase/neon-js/auth/react/ui";
import { authViewPaths } from "@neondatabase/neon-js/auth/react/ui/server";
// CSS (choose one)
import "@neondatabase/neon-js/ui/css"; // Without Tailwind
// @import '@neondatabase/neon-js/ui/tailwind'; // With Tailwind v4 (in CSS file)
```
## Generate Types
```bash
npx neon-js gen-types --db-url "postgresql://..." --output src/types/database.ts
```
## Common Mistakes
1. **Wrong adapter import**: Import `BetterAuthReactAdapter` from `auth/react/adapters` subpath
2. **Forgetting to call adapter**: Use `SupabaseAuthAdapter()` with parentheses
3. **Missing CSS import**: Import from `ui/css` or `ui/tailwind` (not both)
4. **Wrong package for auth-only**: Use `@neondatabase/auth` for smaller bundle
5. **Missing "use client"**: Required for auth client components
See `neon-js/common-mistakes.md` for detailed examples.
@@ -0,0 +1,127 @@
# Neon JS - Common Mistakes
Reference guide for common mistakes when using `@neondatabase/neon-js`.
## Import Mistakes
### BetterAuthReactAdapter Subpath Requirement
`BetterAuthReactAdapter` is **NOT** exported from the main package entry.
**Wrong:**
```typescript
import { BetterAuthReactAdapter } from "@neondatabase/neon-js";
```
**Correct:**
```typescript
import { BetterAuthReactAdapter } from "@neondatabase/neon-js/auth/react/adapters";
```
### Adapter Factory Functions
All adapters must be called with `()`.
**Wrong:**
```typescript
const client = createClient({
auth: {
adapter: BetterAuthReactAdapter, // Missing ()
url: process.env.NEON_AUTH_URL!,
},
dataApi: { url: process.env.NEON_DATA_API_URL! },
});
```
**Correct:**
```typescript
const client = createClient({
auth: {
adapter: BetterAuthReactAdapter(), // Called as function
url: process.env.NEON_AUTH_URL!,
},
dataApi: { url: process.env.NEON_DATA_API_URL! },
});
```
---
## CSS Import Mistakes
Choose **ONE** CSS import method:
**With Tailwind v4:**
```css
@import "tailwindcss";
@import "@neondatabase/neon-js/ui/tailwind";
```
**Without Tailwind:**
```typescript
import "@neondatabase/neon-js/ui/css";
```
**Never import both** - causes duplicate styles.
---
## Environment Variables
**Required for Next.js:**
```bash
# .env.local
NEON_AUTH_BASE_URL=https://ep-xxx.neonauth.c-2.us-east-2.aws.neon.build/dbname/auth
NEXT_PUBLIC_NEON_AUTH_URL=https://ep-xxx.neonauth.c-2.us-east-2.aws.neon.build/dbname/auth
NEON_DATA_API_URL=https://ep-xxx.apirest.c-2.us-east-2.aws.neon.build/dbname/rest/v1
```
**Required for Vite/React SPA:**
```bash
# .env
VITE_NEON_AUTH_URL=https://ep-xxx.neonauth.c-2.us-east-2.aws.neon.build/dbname/auth
VITE_NEON_DATA_API_URL=https://ep-xxx.apirest.c-2.us-east-2.aws.neon.build/dbname/rest/v1
```
---
## Usage Mistakes
### Missing "use client" Directive
```typescript
"use client"; // Required!
import { authClient } from "@/lib/auth/client";
function AuthStatus() {
const session = authClient.useSession();
// ...
}
```
### Wrong API for Adapter
| Adapter | Sign In | Sign Up |
| ---------------------- | ----------------------------------------- | ----------------------------------- |
| BetterAuthReactAdapter | `signIn.email({ email, password })` | `signUp.email({ email, password })` |
| SupabaseAuthAdapter | `signInWithPassword({ email, password })` | `signUp({ email, password })` |
### Using neon-js for Auth Only
If you only need auth (no database queries), use `@neondatabase/auth` for smaller bundle size:
```bash
# Auth only - smaller bundle
npm install @neondatabase/auth
# Auth + Data API - full SDK
npm install @neondatabase/neon-js
```
@@ -0,0 +1,481 @@
# Neon JS Data API Reference
Complete reference for PostgREST-style database queries using `@neondatabase/neon-js`.
## Client Setup
### Next.js
```typescript
// lib/db/client.ts
import { createClient } from "@neondatabase/neon-js";
import type { Database } from "./database.types";
export const dbClient = createClient<Database>({
auth: { url: process.env.NEXT_PUBLIC_NEON_AUTH_URL! },
dataApi: { url: process.env.NEON_DATA_API_URL! },
});
```
### React SPA
```typescript
import { createClient } from "@neondatabase/neon-js";
import { BetterAuthReactAdapter } from "@neondatabase/neon-js/auth/react/adapters";
const client = createClient<Database>({
auth: {
adapter: BetterAuthReactAdapter(),
url: import.meta.env.VITE_NEON_AUTH_URL,
},
dataApi: { url: import.meta.env.VITE_NEON_DATA_API_URL },
});
```
### Node.js Backend
```typescript
import { createClient } from "@neondatabase/neon-js";
const client = createClient<Database>({
auth: { url: process.env.NEON_AUTH_URL! },
dataApi: { url: process.env.NEON_DATA_API_URL! },
});
```
---
## Query Patterns
All query methods follow PostgREST syntax (same as Supabase).
### Select Queries
**Basic select:**
```typescript
const { data, error } = await client.from("items").select();
```
**Select specific columns:**
```typescript
const { data } = await client.from("items").select("id, name, status");
```
**Select with filters:**
```typescript
const { data } = await client
.from("items")
.select("id, name, status")
.eq("status", "active")
.order("created_at", { ascending: false })
.limit(10);
```
**Select single row:**
```typescript
const { data, error } = await client
.from("items")
.select("*")
.eq("id", 1)
.single();
```
### Insert
**Insert single row:**
```typescript
const { data, error } = await client
.from("items")
.insert({ name: "New Item", status: "pending" })
.select()
.single();
```
**Insert multiple rows:**
```typescript
const { data, error } = await client
.from("items")
.insert([
{ name: "Item 1", status: "pending" },
{ name: "Item 2", status: "pending" },
])
.select();
```
### Update
**Update with filter:**
```typescript
await client.from("items").update({ status: "completed" }).eq("id", 1);
```
**Update and return data:**
```typescript
const { data, error } = await client
.from("items")
.update({ status: "completed" })
.eq("id", 1)
.select()
.single();
```
### Delete
**Delete single row:**
```typescript
await client.from("items").delete().eq("id", 1);
```
**Delete and return data:**
```typescript
const { data, error } = await client
.from("items")
.delete()
.eq("id", 1)
.select()
.single();
```
### Upsert
```typescript
await client
.from("items")
.upsert({ id: 1, name: "Updated Item", status: "active" });
```
---
## Filtering
### Comparison Operators
```typescript
// Equal
.eq("status", "active")
// Not equal
.neq("status", "archived")
// Greater than
.gt("price", 100)
// Greater than or equal
.gte("price", 100)
// Less than
.lt("price", 100)
// Less than or equal
.lte("price", 100)
// Like (pattern matching)
.like("name", "%item%")
// ILike (case-insensitive)
.ilike("name", "%item%")
// Is null
.is("deleted_at", null)
// Is not null
.not("deleted_at", "is", null)
// In array
.in("status", ["active", "pending"])
// Contains (for arrays/JSONB)
.contains("tags", ["important"])
```
### Logical Operators
```typescript
// AND (chained)
.eq("status", "active")
.gt("price", 100)
// OR
.or("status.eq.active,price.gt.100")
// NOT
.not("status", "eq", "archived")
```
### Ordering
```typescript
// Ascending
.order("created_at", { ascending: true })
// Descending
.order("created_at", { ascending: false })
// Multiple columns
.order("status", { ascending: true })
.order("created_at", { ascending: false })
```
### Pagination
```typescript
// Limit
.limit(10)
// Range (offset + limit)
.range(0, 9) // First 10 items
// Range for pagination
const page = 1;
const pageSize = 10;
.range((page - 1) * pageSize, page * pageSize - 1)
```
---
## Relationships
### Select with Relationships
**One-to-many:**
```typescript
const { data } = await client
.from("posts")
.select("id, title, author:users(name, email)");
```
**Many-to-many:**
```typescript
const { data } = await client
.from("posts")
.select("id, title, tags:post_tags(tag:tags(name))");
```
**Nested relationships:**
```typescript
const { data } = await client.from("posts").select(`
id,
title,
author:users(
id,
name,
profile:profiles(bio, avatar)
)
`);
```
---
## Type Generation
Generate TypeScript types from your database schema:
```bash
npx neon-js gen-types --db-url "postgresql://user:pass@host/db" --output src/types/database.ts
```
Or using environment variable:
```bash
npx neon-js gen-types --db-url "$DATABASE_URL" --output lib/db/database.types.ts
```
**Use types in client:**
```typescript
import { createClient } from "@neondatabase/neon-js";
import type { Database } from "./database.types";
export const dbClient = createClient<Database>({
auth: { url: process.env.NEXT_PUBLIC_NEON_AUTH_URL! },
dataApi: { url: process.env.NEON_DATA_API_URL! },
});
```
**Benefits:**
- Full TypeScript autocomplete for tables and columns
- Type-safe queries
- Compile-time error checking
---
## Error Handling
**Check for errors:**
```typescript
const { data, error } = await client.from("items").select();
if (error) {
console.error("Database error:", error.message);
console.error("Error code:", error.code);
console.error("Error details:", error.details);
return;
}
// Use data
console.log(data);
```
**Common error codes:**
- `PGRST116` - No rows returned (when using `.single()`)
- `23505` - Unique violation
- `23503` - Foreign key violation
- `42P01` - Table does not exist
---
## Usage Examples
### Server Component (Next.js)
```typescript
// app/posts/page.tsx
import { dbClient } from "@/lib/db/client";
export default async function PostsPage() {
const { data: posts, error } = await dbClient
.from("posts")
.select("id, title, created_at, author:users(name)")
.order("created_at", { ascending: false })
.limit(10);
if (error) return <div>Error loading posts</div>;
return (
<ul>
{posts?.map((post) => (
<li key={post.id}>
<h2>{post.title}</h2>
<p>By {post.author?.name}</p>
</li>
))}
</ul>
);
}
```
### API Route (Next.js)
```typescript
// app/api/posts/route.ts
import { dbClient } from "@/lib/db/client";
import { NextResponse } from "next/server";
export async function GET() {
const { data, error } = await dbClient.from("posts").select();
if (error) {
return NextResponse.json({ error: error.message }, { status: 500 });
}
return NextResponse.json(data);
}
export async function POST(request: Request) {
const body = await request.json();
const { data, error } = await dbClient
.from("posts")
.insert(body)
.select()
.single();
if (error) {
return NextResponse.json({ error: error.message }, { status: 400 });
}
return NextResponse.json(data, { status: 201 });
}
```
### Client Component (React)
```typescript
"use client";
import { useEffect, useState } from "react";
import { dbClient } from "@/lib/db/client";
export function ItemsList() {
const [items, setItems] = useState([]);
const [loading, setLoading] = useState(true);
useEffect(() => {
async function fetchItems() {
const { data, error } = await dbClient
.from("items")
.select("id, name, status")
.eq("status", "active");
if (error) {
console.error(error);
return;
}
setItems(data || []);
setLoading(false);
}
fetchItems();
}, []);
if (loading) return <div>Loading...</div>;
return (
<ul>
{items.map((item) => (
<li key={item.id}>{item.name}</li>
))}
</ul>
);
}
```
---
## Supabase Migration
The Neon JS SDK uses the same PostgREST API as Supabase, making migration straightforward:
**Before (Supabase):**
```typescript
import { createClient } from "@supabase/supabase-js";
const client = createClient(SUPABASE_URL, SUPABASE_KEY);
```
**After (Neon):**
```typescript
import { createClient, SupabaseAuthAdapter } from "@neondatabase/neon-js";
const client = createClient({
auth: { adapter: SupabaseAuthAdapter(), url: NEON_AUTH_URL },
dataApi: { url: NEON_DATA_API_URL },
});
```
**Query syntax remains the same:**
```typescript
// Works identically in both
await client.auth.signInWithPassword({ email, password });
const { data } = await client.from("items").select();
```
@@ -0,0 +1,96 @@
# Neon Platform API
The Neon Platform API allows you to manage Neon projects, branches, databases, and resources programmatically. You can use the REST API directly or through official SDKs.
## Options
| Method | Package/URL | Best For |
| -------------- | ----------------------------------- | ------------------------------- |
| REST API | `https://console.neon.tech/api/v2/` | Any language, direct HTTP calls |
| TypeScript SDK | `@neondatabase/api-client` | Node.js, TypeScript projects |
| Python SDK | `neon-api` | Python scripts and applications |
| CLI | `neonctl` | Terminal-based management |
## Documentation
```bash
# REST API documentation
curl -H "Accept: text/markdown" https://neon.tech/docs/reference/api-reference
# TypeScript SDK
curl -H "Accept: text/markdown" https://neon.tech/docs/reference/typescript-sdk
# Python SDK
curl -H "Accept: text/markdown" https://neon.tech/docs/reference/python-sdk
# CLI
curl -H "Accept: text/markdown" https://neon.tech/docs/reference/neon-cli
```
For the interactive API reference: https://api-docs.neon.tech/reference/getting-started-with-neon-api
## Sub-Resources
For detailed information, reference the appropriate sub-resource:
### REST API Details
| Topic | Resource |
| ----------------------------- | -------------------------------- |
| Guidelines, Auth, Rate Limits | `neon-rest-api/guidelines.md` |
| Projects | `neon-rest-api/projects.md` |
| Branches, Databases, Roles | `neon-rest-api/branches.md` |
| Compute Endpoints | `neon-rest-api/endpoints.md` |
| API Keys | `neon-rest-api/keys.md` |
| Operations | `neon-rest-api/operations.md` |
| Organizations | `neon-rest-api/organizations.md` |
### SDKs
| Language | Resource |
| ---------- | ------------------------ |
| TypeScript | `neon-typescript-sdk.md` |
| Python | `neon-python-sdk.md` |
## Quick Start
### Authentication
All API requests require a Neon API key:
```bash
Authorization: Bearer $NEON_API_KEY
```
### API Key Types
| Type | Scope | Best For |
| -------------- | ------------------------------- | ----------------------------- |
| Personal | All projects user has access to | Individual use, scripting |
| Organization | Entire organization | CI/CD, org-wide automation |
| Project-scoped | Single project only | Project-specific integrations |
### Rate Limits
- 700 requests per minute (~11 per second)
- Bursts up to 40 requests per second per route
- Handle `429 Too Many Requests` with retry/backoff
## Common Operations Quick Reference
| Operation | REST API | TypeScript SDK | Python SDK |
| ------------------ | ------------------------------------------ | --------------------------- | --------------------- |
| List Projects | `GET /projects` | `listProjects({})` | `projects()` |
| Create Project | `POST /projects` | `createProject({...})` | `project_create(...)` |
| Get Connection URI | `GET /projects/{id}/connection_uri` | `getConnectionUri({...})` | `connection_uri(...)` |
| Create Branch | `POST /projects/{id}/branches` | `createProjectBranch(...)` | `branch_create(...)` |
| Start Endpoint | `POST /projects/{id}/endpoints/{id}/start` | `startProjectEndpoint(...)` | `endpoint_start(...)` |
## Error Handling
| Status | Meaning | Action |
| ------ | ------------ | ---------------------------- |
| 401 | Unauthorized | Check API key |
| 404 | Not Found | Verify resource ID |
| 429 | Rate Limited | Implement retry with backoff |
| 500 | Server Error | Retry or contact support |
@@ -0,0 +1,261 @@
# Neon Python SDK
The `neon-api` Python SDK is a Pythonic wrapper around the Neon REST API. It provides methods for managing all Neon resources, including projects, branches, endpoints, roles, and databases.
For core concepts (Organization, Project, Branch, Endpoint, etc.), see `what-is-neon.md`.
## Documentation
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/reference/python-sdk
```
## Installation
```bash
pip install neon-api
```
## Authentication
```python
import os
from neon_api import NeonAPI
api_key = os.getenv("NEON_API_KEY")
if not api_key:
raise ValueError("NEON_API_KEY environment variable is not set.")
neon = NeonAPI(api_key=api_key)
```
## Projects
### List Projects
```python
all_projects = neon.projects()
```
### Create Project
```python
new_project = neon.project_create(
project={
'name': 'my-new-project',
'pg_version': 17
}
)
```
### Get Project Details
```python
project = neon.project(project_id='your-project-id')
```
### Update Project
```python
neon.project_update(
project_id='your-project-id',
project={
'name': 'renamed-project',
'default_endpoint_settings': {
'autoscaling_limit_min_cu': 1,
'autoscaling_limit_max_cu': 2,
}
}
)
```
### Delete Project
```python
neon.project_delete(project_id='project-to-delete')
```
### Get Connection URI
```python
uri = neon.connection_uri(
project_id='your-project-id',
database_name='neondb',
role_name='neondb_owner'
)
print(f"Connection URI: {uri.uri}")
```
## Branches
### Create Branch
```python
new_branch = neon.branch_create(
project_id='your-project-id',
branch={'name': 'feature-branch'},
endpoints=[
{'type': 'read_write', 'autoscaling_limit_max_cu': 1}
]
)
```
### List Branches
```python
branches = neon.branches(project_id='your-project-id')
```
### Get Branch Details
```python
branch = neon.branch(project_id='your-project-id', branch_id='br-xxx')
```
### Update Branch
```python
neon.branch_update(
project_id='your-project-id',
branch_id='br-xxx',
branch={'name': 'updated-branch-name'}
)
```
### Delete Branch
```python
neon.branch_delete(project_id='your-project-id', branch_id='br-xxx')
```
## Databases
### Create Database
```python
neon.database_create(
project_id='your-project-id',
branch_id='br-xxx',
database={'name': 'my-app-db', 'owner_name': 'neondb_owner'}
)
```
### List Databases
```python
databases = neon.databases(project_id='your-project-id', branch_id='br-xxx')
```
### Delete Database
```python
neon.database_delete(
project_id='your-project-id',
branch_id='br-xxx',
database_id='my-app-db'
)
```
## Roles
### Create Role
```python
new_role = neon.role_create(
project_id='your-project-id',
branch_id='br-xxx',
role_name='app_user'
)
print(f"Password: {new_role.role.password}")
```
### List Roles
```python
roles = neon.roles(project_id='your-project-id', branch_id='br-xxx')
```
### Delete Role
```python
neon.role_delete(
project_id='your-project-id',
branch_id='br-xxx',
role_name='app_user'
)
```
## Endpoints
### Create Endpoint
```python
neon.endpoint_create(
project_id='your-project-id',
endpoint={
'branch_id': 'br-xxx',
'type': 'read_only'
}
)
```
### Start/Suspend Endpoint
```python
# Start
neon.endpoint_start(project_id='your-project-id', endpoint_id='ep-xxx')
# Suspend
neon.endpoint_suspend(project_id='your-project-id', endpoint_id='ep-xxx')
```
### Update Endpoint
```python
neon.endpoint_update(
project_id='your-project-id',
endpoint_id='ep-xxx',
endpoint={'autoscaling_limit_max_cu': 2}
)
```
### Delete Endpoint
```python
neon.endpoint_delete(project_id='your-project-id', endpoint_id='ep-xxx')
```
## API Keys
### List API Keys
```python
api_keys = neon.api_keys()
```
### Create API Key
```python
new_key = neon.api_key_create(key_name='my-script-key')
print(f"Key (store securely!): {new_key.key}")
```
### Revoke API Key
```python
neon.api_key_revoke(1234) # key ID
```
## Operations
### List Operations
```python
ops = neon.operations(project_id='your-project-id')
```
### Get Operation Details
```python
op = neon.operation(project_id='your-project-id', operation_id='op-xxx')
```
@@ -0,0 +1,552 @@
## Overview
This document outlines the rules for managing branches in a Neon project using the Neon API.
## Manage branches
### Create branch
1. Action: Creates a new branch within a specified project. By default, a branch is created from the project's default branch, but you can specify a parent branch, a point-in-time (LSN or timestamp), and attach compute endpoints.
2. Endpoint: `POST /projects/{project_id}/branches`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project where the branch will be created.
4. Body Parameters: The request body is optional. If provided, it can contain `endpoints` and/or `branch` objects.
`endpoints` (array of objects, optional): A list of compute endpoints to create and attach to the new branch.
- `type` (string, required): The endpoint type. Allowed values: `read_write`, `read_only`.
- `autoscaling_limit_min_cu` (number, optional): The minimum number of Compute Units (CU). Minimum value is `0.25`.
- `autoscaling_limit_max_cu` (number, optional): The maximum number of Compute Units (CU). Minimum value is `0.25`.
- `provisioner` (string, optional): The compute provisioner. Specify `k8s-neonvm` to enable Autoscaling. Allowed values: `k8s-pod`, `k8s-neonvm`.
- `suspend_timeout_seconds` (integer, optional): Duration of inactivity in seconds before a compute is suspended. Ranges from -1 (never suspend) to 604800 (1 week). A value of `0` uses the default of 300 seconds (5 minutes).
`branch` (object, optional): Specifies the properties of the new branch.
- `name` (string, optional): A name for the branch (max 256 characters). If omitted, a name is auto-generated.
- `parent_id` (string, optional): The ID of the parent branch. If omitted, the project's default branch is used as the parent.
- `parent_lsn` (string, optional): A Log Sequence Number (LSN) from the parent branch to create the new branch from a specific point-in-time.
- `parent_timestamp` (string, optional): An ISO 8601 timestamp (e.g., `2025-08-26T12:00:00Z`) to create the branch from a specific point-in-time.
- `protected` (boolean, optional): If `true`, the branch is created as a protected branch.
- `init_source` (string, optional): The source for branch initialization. `parent-data` (default) copies schema and data. `schema-only` creates a new root branch with only the schema from the specified parent.
- `expires_at` (string, optional): An RFC 3339 timestamp for when the branch should be automatically deleted (e.g., `2025-06-09T18:02:16Z`).
Example: Create a branch from a specific parent with a read-write compute
```bash
curl 'https://console.neon.tech/api/v2/projects/hidden-river-50598307/branches' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY" \
-H 'Content-Type: application/json' \
-d '{
"endpoints": [
{
"type": "read_write"
}
],
"branch": {
"parent_id": "br-super-wildflower-adniii9u",
"name": "my-new-feature-branch"
}
}'
```
Example response
```json
{
"branch": {
"id": "br-damp-glitter-adqd4hk5",
"project_id": "hidden-river-50598307",
"parent_id": "br-super-wildflower-adniii9u",
"parent_lsn": "0/1A7F730",
"name": "my-new-feature-branch",
"current_state": "init",
"pending_state": "ready",
"state_changed_at": "2025-09-10T16:45:52Z",
"creation_source": "console",
"primary": false,
"default": false,
"protected": false,
"cpu_used_sec": 0,
"compute_time_seconds": 0,
"active_time_seconds": 0,
"written_data_bytes": 0,
"data_transfer_bytes": 0,
"created_at": "2025-09-10T16:45:52Z",
"updated_at": "2025-09-10T16:45:52Z",
"created_by": {
"name": "<USER_NAME>",
"image": "<USER_IMAGE_URL>"
},
"init_source": "parent-data"
},
"endpoints": [
{
"host": "ep-raspy-glade-ad8e3gvy.c-2.us-east-1.aws.neon.tech",
"id": "ep-raspy-glade-ad8e3gvy",
"project_id": "hidden-river-50598307",
"branch_id": "br-damp-glitter-adqd4hk5",
"autoscaling_limit_min_cu": 0.25,
"autoscaling_limit_max_cu": 2,
"region_id": "aws-us-east-1",
"type": "read_write",
"current_state": "init",
"pending_state": "active",
"settings": {},
"pooler_enabled": false,
"pooler_mode": "transaction",
"disabled": false,
"passwordless_access": true,
"creation_source": "console",
"created_at": "2025-09-10T16:45:52Z",
"updated_at": "2025-09-10T16:45:52Z",
"proxy_host": "c-2.us-east-1.aws.neon.tech",
"suspend_timeout_seconds": 0,
"provisioner": "k8s-neonvm"
}
],
"operations": [
{
"id": "cf5d0923-fc13-4125-83d5-8fc31c6b0214",
"project_id": "hidden-river-50598307",
"branch_id": "br-damp-glitter-adqd4hk5",
"action": "create_branch",
"status": "running",
"failures_count": 0,
"created_at": "2025-09-10T16:45:52Z",
"updated_at": "2025-09-10T16:45:52Z",
"total_duration_ms": 0
},
{
"id": "e3c60b62-00c8-4ad4-9cd1-cdc3e8fd8154",
"project_id": "hidden-river-50598307",
"branch_id": "br-damp-glitter-adqd4hk5",
"endpoint_id": "ep-raspy-glade-ad8e3gvy",
"action": "start_compute",
"status": "scheduling",
"failures_count": 0,
"created_at": "2025-09-10T16:45:52Z",
"updated_at": "2025-09-10T16:45:52Z",
"total_duration_ms": 0
}
],
"roles": [
{
"branch_id": "br-damp-glitter-adqd4hk5",
"name": "neondb_owner",
"protected": false,
"created_at": "2025-09-10T12:14:58Z",
"updated_at": "2025-09-10T12:14:58Z"
}
],
"databases": [
{
"id": 9554148,
"branch_id": "br-damp-glitter-adqd4hk5",
"name": "neondb",
"owner_name": "neondb_owner",
"created_at": "2025-09-10T12:14:58Z",
"updated_at": "2025-09-10T12:14:58Z"
}
],
"connection_uris": [
{
"connection_uri": "postgresql://neondb_owner:npg_EwcS9IOgFfb7@ep-raspy-glade-ad8e3gvy.c-2.us-east-1.aws.neon.tech/neondb?sslmode=require",
"connection_parameters": {
"database": "neondb",
"password": "npg_EwcS9IOgFfb7",
"role": "neondb_owner",
"host": "ep-raspy-glade-ad8e3gvy.c-2.us-east-1.aws.neon.tech",
"pooler_host": "ep-raspy-glade-ad8e3gvy-pooler.c-2.us-east-1.aws.neon.tech"
}
}
]
}
```
### List branches
1. Action: Retrieves a list of branches for the specified project. Supports filtering, sorting, and pagination.
2. Endpoint: `GET /projects/{project_id}/branches`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
4. Query Parameters:
- `search` (string, optional): Filters branches by a partial match on name or ID.
- `sort_by` (string, optional): The field to sort by. Allowed values: `name`, `created_at`, `updated_at`. Defaults to `updated_at`.
- `sort_order` (string, optional): The sort order. Allowed values: `asc`, `desc`. Defaults to `desc`.
- `limit` (integer, optional): The number of branches to return (1 to 10000).
- `cursor` (string, optional): The cursor from a previous response for pagination.
Example: List all branches sorted by creation date
```bash
curl 'https://console.neon.tech/api/v2/projects/hidden-river-50598307/branches?sort_by=created_at&sort_order=asc' \
-H 'accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
Example response
```json
{
"branches": [
{
"id": "br-long-feather-adpbgzlx",
"project_id": "hidden-river-50598307",
"name": "production",
"current_state": "ready",
"state_changed_at": "2025-09-10T12:15:01Z",
"logical_size": 30785536,
"creation_source": "console",
"primary": true,
"default": true,
"protected": false,
"cpu_used_sec": 82,
"compute_time_seconds": 82,
"active_time_seconds": 316,
"written_data_bytes": 29060360,
"data_transfer_bytes": 0,
"created_at": "2025-09-10T12:14:58Z",
"updated_at": "2025-09-10T12:35:33Z",
"created_by": {
"name": "<USER_NAME>",
"image": "<USER_IMAGE_URL>"
},
"init_source": "parent-data"
},
{
"id": "br-super-wildflower-adniii9u",
"project_id": "hidden-river-50598307",
"parent_id": "br-long-feather-adpbgzlx",
"parent_lsn": "0/1A33BC8",
"parent_timestamp": "2025-09-10T12:15:03Z",
"name": "development",
"current_state": "ready",
"state_changed_at": "2025-09-10T12:15:04Z",
"logical_size": 30842880,
"creation_source": "console",
"primary": false,
"default": false,
"protected": false,
"cpu_used_sec": 78,
"compute_time_seconds": 78,
"active_time_seconds": 312,
"written_data_bytes": 310120,
"data_transfer_bytes": 0,
"created_at": "2025-09-10T12:15:04Z",
"updated_at": "2025-09-10T12:35:33Z",
"created_by": {
"name": "<USER_NAME>",
"image": "<USER_IMAGE_URL>"
},
"init_source": "parent-data"
}
],
"annotations": {
"br-long-feather-adpbgzlx": {
"object": {
"type": "console/branch",
"id": "br-long-feather-adpbgzlx"
},
"value": {
"environment": "production"
},
"created_at": "2025-09-10T12:14:58Z",
"updated_at": "2025-09-10T12:14:58Z"
}
},
"pagination": {
"sort_by": "created_at",
"sort_order": "ASC"
}
}
```
### Retrieve branch details
1. Action: Retrieves detailed information about a specific branch, including its parent, creation timestamp, and state.
2. Endpoint: `GET /projects/{project_id}/branches/{branch_id}`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
- `branch_id` (string, required): The unique identifier of the branch.
Example Request:
```bash
curl 'https://console.neon.tech/api/v2/projects/hidden-river-50598307/branches/br-super-wildflower-adniii9u' \
-H 'accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
Example Response:
```json
{
"branch": {
"id": "br-super-wildflower-adniii9u",
"project_id": "hidden-river-50598307",
"parent_id": "br-long-feather-adpbgzlx",
"parent_lsn": "0/1A33BC8",
"parent_timestamp": "2025-09-10T12:15:03Z",
"name": "development",
"current_state": "ready",
"state_changed_at": "2025-09-10T12:15:04Z",
"logical_size": 30842880,
"creation_source": "console",
"primary": false,
"default": false,
"protected": false,
"cpu_used_sec": 78,
"compute_time_seconds": 78,
"active_time_seconds": 312,
"written_data_bytes": 310120,
"data_transfer_bytes": 0,
"created_at": "2025-09-10T12:15:04Z",
"updated_at": "2025-09-10T12:35:33Z",
"created_by": {
"name": "<USER_NAME>",
"image": "<USER_IMAGE_URL>"
},
"init_source": "parent-data"
},
"annotation": {
"object": {
"type": "console/branch",
"id": "br-super-wildflower-adniii9u"
},
"value": {
"environment": "development"
},
"created_at": "2025-09-10T12:15:04Z",
"updated_at": "2025-09-10T12:15:04Z"
}
}
```
### Update branch
1. Action: Updates the properties of a specified branch, such as its name, protection status, or expiration time.
2. Endpoint: `PATCH /projects/{project_id}/branches/{branch_id}`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
- `branch_id` (string, required): The unique identifier of the branch to update.
4. Body Parameters:
`branch` (object, required): The container for the branch attributes to update.
- `name` (string, optional): A new name for the branch (max 256 characters).
- `protected` (boolean, optional): Set to `true` to protect the branch or `false` to unprotect it.
- `expires_at` (string or null, optional): Set a new RFC 3339 expiration timestamp or `null` to remove the expiration.
Example: Change branch name:
```bash
curl -X 'PATCH' \
'https://console.neon.tech/api/v2/projects/hidden-river-50598307/branches/br-damp-glitter-adqd4hk5' \
-H 'accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY" \
-H 'Content-Type: application/json' \
-d '{
"branch": {
"name": "updated-branch-name"
}
}'
```
Example response:
```json
{
"branch": {
"id": "br-damp-glitter-adqd4hk5",
"project_id": "hidden-river-50598307",
"parent_id": "br-super-wildflower-adniii9u",
"parent_lsn": "0/1A7F730",
"parent_timestamp": "2025-09-10T12:15:05Z",
"name": "updated-branch-name",
"current_state": "ready",
"state_changed_at": "2025-09-10T16:45:52Z",
"logical_size": 30842880,
"creation_source": "console",
"primary": false,
"default": false,
"protected": false,
"cpu_used_sec": 68,
"compute_time_seconds": 68,
"active_time_seconds": 268,
"written_data_bytes": 0,
"data_transfer_bytes": 0,
"created_at": "2025-09-10T16:45:52Z",
"updated_at": "2025-09-10T16:55:30Z",
"created_by": {
"name": "<USER_NAME>",
"image": "<USER_IMAGE_URL>"
},
"init_source": "parent-data"
},
"operations": []
}
```
### Delete branch
1. Action: Deletes the specified branch from a project. This action will also place all associated compute endpoints into an idle state, breaking any active client connections.
2. Endpoint: `DELETE /projects/{project_id}/branches/{branch_id}`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
- `branch_id` (string, required): The unique identifier of the branch to delete.
4. Constraints:
- You cannot delete a project's root or default branch.
- You cannot delete a branch that has child branches. You must delete all child branches first.
Example Request:
```bash
curl -X 'DELETE' \
'https://console.neon.tech/api/v2/projects/{project_id}/branches/{branch_id}' \
-H 'accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
### List branch endpoints
1. Action: Retrieves a list of all compute endpoints that are associated with a specific branch.
2. Endpoint: `GET /projects/{project_id}/branches/{branch_id}/endpoints`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
- `branch_id` (string, required): The unique identifier of the branch whose endpoints you want to list.
4. A branch can have one `read_write` compute endpoint and multiple `read_only` endpoints. This method returns an array of all endpoints currently attached to the specified branch.
Example Request:
```bash
curl 'https://console.neon.tech/api/v2/projects/hidden-river-50598307/branches/br-super-wildflower-adniii9u/endpoints' \
-H 'accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
## Manage databases
### Create database
1. Action: Creates a new database within a specified branch. A branch can contain multiple databases.
2. Endpoint: `POST /projects/{project_id}/branches/{branch_id}/databases`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
- `branch_id` (string, required): The unique identifier of the branch where the database will be created.
4. Body Parameters:
`database` (object, required): The container for the new database's properties.
- `name` (string, required): The name for the new database.
- `owner_name` (string, required): The name of an existing role that will own the database.
Example Request:
```bash
curl 'https://console.neon.tech/api/v2/projects/hidden-river-50598307/branches/br-super-wildflower-adniii9u/databases' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY" \
-H 'Content-Type: application/json' \
-d '{
"database": {
"name": "my_new_app_db",
"owner_name": "app_owner_role"
}
}'
```
### List databases
1. Action: Retrieves a list of all databases within a specified branch.
2. Endpoint: `GET /projects/{project_id}/branches/{branch_id}/databases`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
- `branch_id` (string, required): The unique identifier of the branch.
Example Request:
```bash
curl 'https://console.neon.tech/api/v2/projects/hidden-river-50598307/branches/br-super-wildflower-adniii9u/databases' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
### Retrieve database details
1. Action: Retrieves detailed information about a specific database within a branch.
2. Endpoint: `GET /projects/{project_id}/branches/{branch_id}/databases/{database_name}`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
- `branch_id` (string, required): The unique identifier of the branch.
- `database_name` (string, required): The name of the database.
### Update database
1. Action: Updates the properties of a specified database, such as its name or owner.
2. Endpoint: `PATCH /projects/{project_id}/branches/{branch_id}/databases/{database_name}`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
- `branch_id` (string, required): The unique identifier of the branch.
- `database_name` (string, required): The current name of the database to update.
4. Body Parameters:
`database` (object, required): The container for the database attributes to update.
- `name` (string, optional): A new name for the database.
- `owner_name` (string, optional): The name of a different existing role to become the new owner.
### Delete database
1. Action: Deletes the specified database from a branch. This action is permanent and cannot be undone.
2. Endpoint: `DELETE /projects/{project_id}/branches/{branch_id}/databases/{database_name}`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
- `branch_id` (string, required): The unique identifier of the branch.
- `database_name` (string, required): The name of the database to delete.
## Manage roles
### Create role
1. Action: Creates a new Postgres role in a specified branch. This action may drop existing connections to the active compute endpoint.
2. Endpoint: `POST /projects/{project_id}/branches/{branch_id}/roles`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
- `branch_id` (string, required): The unique identifier of the branch where the role will be created.
4. Body Parameters:
`role` (object, required): The container for the new role's properties.
- `name` (string, required): The name for the new role. Cannot exceed 63 bytes in length.
- `no_login` (boolean, optional): If `true`, creates a role that cannot be used to log in. Defaults to `false`.
Example Request:
```bash
curl 'https://console.neon.tech/api/v2/projects/hidden-river-50598307/branches/br-super-wildflower-adniii9u/roles' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY" \
-H 'Content-Type: application/json' \
-d '{
"role": {
"name": "new_app_user"
}
}'
```
### List roles
1. Action: Retrieves a list of all Postgres roles from the specified branch.
2. Endpoint: `GET /projects/{project_id}/branches/{branch_id}/roles`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
- `branch_id` (string, required): The unique identifier of the branch.
### Retrieve role details
1. Action: Retrieves detailed information about a specific Postgres role within a branch.
2. Endpoint: `GET /projects/{project_id}/branches/{branch_id}/roles/{role_name}`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
- `branch_id` (string, required): The unique identifier of the branch.
- `role_name` (string, required): The name of the role.
### Delete role
1. Action: Deletes the specified Postgres role from the branch. This action is permanent.
2. Endpoint: `DELETE /projects/{project_id}/branches/{branch_id}/roles/{role_name}`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
- `branch_id` (string, required): The unique identifier of the branch.
- `role_name` (string, required): The name of the role to delete.
@@ -0,0 +1,211 @@
## Overview
This section provides rules for managing compute endpoints associated with branches in a project. Compute endpoints are Neon compute instances that allow you to connect to and interact with your databases.
## Manage compute endpoints
### Create compute endpoint
1. Action: Creates a new compute endpoint (a Neon compute instance) and associates it with a specified branch.
2. Endpoint: `POST /projects/{project_id}/endpoints`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
4. Body Parameters:
`endpoint` (object, required): The container for the new endpoint's properties.
- `branch_id` (string, required): The ID of the branch to associate the endpoint with.
- `type` (string, required): The endpoint type. A branch can have only one `read_write` endpoint but multiple `read_only` endpoints. Allowed values: `read_write`, `read_only`.
- `region_id` (string, optional): The region where the endpoint will be created. Must match the project's region.
- `autoscaling_limit_min_cu` (number, optional): The minimum number of Compute Units (CU). Minimum `0.25`.
- `autoscaling_limit_max_cu` (number, optional): The maximum number of Compute Units (CU). Minimum `0.25`.
- `provisioner` (string, optional): The compute provisioner. Specify `k8s-neonvm` to enable Autoscaling. Allowed values: `k8s-pod`, `k8s-neonvm`.
- `suspend_timeout_seconds` (integer, optional): Duration of inactivity in seconds before suspending the compute. Ranges from -1 (never suspend) to 604800 (1 week).
- `disabled` (boolean, optional): If `true`, restricts connections to the endpoint.
Example Request:
```bash
curl 'https://console.neon.tech/api/v2/projects/hidden-river-50598307/endpoints' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY" \
-H 'Content-Type: application/json' \
-d '{
"endpoint": {
"branch_id": "br-your-branch-id",
"type": "read_only"
}
}'
```
Example Response:
```json
{
"endpoint": {
"host": "ep-proud-mud-adwmnxz4.c-2.us-east-1.aws.neon.tech",
"id": "ep-proud-mud-adwmnxz4",
"project_id": "hidden-river-50598307",
"branch_id": "br-super-wildflower-adniii9u",
"autoscaling_limit_min_cu": 0.25,
"autoscaling_limit_max_cu": 2,
"region_id": "aws-us-east-1",
"type": "read_only",
"current_state": "init",
"pending_state": "active",
"settings": {},
"pooler_enabled": false,
"pooler_mode": "transaction",
"disabled": false,
"passwordless_access": true,
"creation_source": "console",
"created_at": "2025-09-11T06:25:12Z",
"updated_at": "2025-09-11T06:25:12Z",
"proxy_host": "c-2.us-east-1.aws.neon.tech",
"suspend_timeout_seconds": 0,
"provisioner": "k8s-neonvm"
},
"operations": [
{
"id": "4d10642f-5212-4517-ad60-afd28c9096e2",
"project_id": "hidden-river-50598307",
"branch_id": "br-super-wildflower-adniii9u",
"endpoint_id": "ep-proud-mud-adwmnxz4",
"action": "start_compute",
"status": "running",
"failures_count": 0,
"created_at": "2025-09-11T06:25:12Z",
"updated_at": "2025-09-11T06:25:12Z",
"total_duration_ms": 0
}
]
}
```
### List compute endpoints
1. Action: Retrieves a list of all compute endpoints for the specified project.
2. Endpoint: `GET /projects/{project_id}/endpoints`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
Example Request:
```bash
curl 'https://console.neon.tech/api/v2/projects/hidden-river-50598307/endpoints' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
### Retrieve compute endpoint details
1. Action: Retrieves detailed information about a specific compute endpoint, including its configuration (e.g., autoscaling limits), current state (`active` or `idle`), and associated branch ID.
2. Endpoint: `GET /projects/{project_id}/endpoints/{endpoint_id}`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
- `endpoint_id` (string, required): The unique identifier of the compute endpoint).
Example Request:
```bash
curl 'https://console.neon.tech/api/v2/projects/hidden-river-50598307/endpoints/ep-proud-mud-adwmnxz4' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
### Update compute endpoint
1. Action: Updates the configuration of a specified compute endpoint.
2. Endpoint: `PATCH /projects/{project_id}/endpoints/{endpoint_id}`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
- `endpoint_id` (string, required): The unique identifier of the compute endpoint.
4. Body Parameters:
`endpoint` (object, required): The container for the endpoint attributes to update.
- `autoscaling_limit_min_cu` (number, optional): A new minimum number of Compute Units (CU).
- `autoscaling_limit_max_cu` (number, optional): A new maximum number of Compute Units (CU).
- `suspend_timeout_seconds` (integer, optional): A new inactivity period in seconds before suspension.
- `disabled` (boolean, optional): Set to `true` to disable connections or `false` to enable them.
- `provisioner` (string, optional): Change the compute provisioner.
Example: Update autoscaling limits
```bash
curl -X 'PATCH' \
'https://console.neon.tech/api/v2/projects/hidden-river-50598307/endpoints/ep-proud-mud-adwmnxz4' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY" \
-H 'Content-Type: application/json' \
-d '{
"endpoint": {
"autoscaling_limit_min_cu": 0.5,
"autoscaling_limit_max_cu": 1
}
}'
```
### Delete compute endpoint
1. Action: Deletes the specified compute endpoint. This action drops any existing network connections to the endpoint.
2. Endpoint: `DELETE /projects/{project_id}/endpoints/{endpoint_id}`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
- `endpoint_id` (string, required): The unique identifier of the compute endpoint to delete.
Example Request:
```bash
curl -X 'DELETE' \
'https://console.neon.tech/api/v2/projects/hidden-river-50598307/endpoints/ep-proud-mud-adwmnxz4' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
### Start compute endpoint
1. Action: Manually starts a compute endpoint that is currently in an `idle` state. The endpoint is ready for connections once the start operation completes successfully.
2. Endpoint: `POST /projects/{project_id}/endpoints/{endpoint_id}/start`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
- `endpoint_id` (string, required): The unique identifier of the compute endpoint.
Example Request:
```bash
curl -X 'POST' \
'https://console.neon.tech/api/v2/projects/hidden-river-50598307/endpoints/ep-ancient-brook-ad5ea04d/start' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
### Suspend compute endpoint
1. Action: Manually suspends an `active` compute endpoint, forcing it into an `idle` state. This will immediately drop any active connections to the endpoint.
2. Endpoint: `POST /projects/{project_id}/endpoints/{endpoint_id}/suspend`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
- `endpoint_id` (string, required): The unique identifier of the compute endpoint.
Example Request:
```bash
curl -X 'POST' \
'https://console.neon.tech/api/v2/projects/hidden-river-50598307/endpoints/ep-ancient-brook-ad5ea04d/suspend' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
### Restart compute endpoint
1. Action: Restarts the specified compute endpoint. This involves an immediate suspend operation followed by a start operation. This is useful for applying configuration changes or refreshing the compute instance. All active connections will be dropped.
2. Endpoint: `POST /projects/{project_id}/endpoints/{endpoint_id}/restart`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project.
- `endpoint_id` (string, required): The unique identifier of the compute endpoint.
Example Request:
```bash
curl -X 'POST' \
'https://console.neon.tech/api/v2/projects/hidden-river-50598307/endpoints/ep-ancient-brook-ad5ea04d/restart' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
@@ -0,0 +1,67 @@
## Overview
This document provides a comprehensive set of rules and guidelines for an AI agent to interact with the Neon API. The Neon API is a RESTful service that allows for programmatic management of all Neon resources. Adherence to these rules ensures correct, efficient, and safe API usage.
### General API guidelines
All Neon API requests must be made to the following base URL:
```
https://console.neon.tech/api/v2/
```
To construct a full request URL, append the specific endpoint path to this base URL.
### Authentication
- All API requests must be authenticated using a Neon API key.
- The API key must be included in the `Authorization` header using the `Bearer` authentication scheme.
- The header should be formatted as: `Authorization: Bearer $NEON_API_KEY`, where `$NEON_API_KEY` is a valid Neon API key.
- A request without a valid `Authorization` header will fail with a `401 Unauthorized` status code.
### API rate limiting
- Neon limits API requests to 700 requests per minute (approximately 11 per second).
- Bursts of up to 40 requests per second per route are permitted.
- If the rate limit is exceeded, the API will respond with an `HTTP 429 Too Many Requests` error.
- Your application logic must handle `429` errors and implement a retry strategy with appropriate backoff.
### Neon Core Concepts
To effectively use the Neon Python SDK, it's essential to understand the hierarchy and purpose of its core resources. The following table provides a high-level overview of each concept.
| Concept | Description | Analogy/Purpose | Key Relationship |
| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
| Organization | The highest-level container, managing billing, users, and multiple projects. | A GitHub Organization or a company's cloud account. | Contains one or more Projects. |
| Project | The primary container that contains all related database resources for a single application or service. | A Git repository or a top-level folder for an application. | Lives within an Organization (or a personal account). Contains Branches. |
| Branch | A lightweight, copy-on-write clone of a database's state at a specific point in time. | A `git branch`. Used for isolated development, testing, staging, or previews without duplicating storage costs. | Belongs to a Project. Contains its own set of Databases and Roles, cloned from its parent. |
| Compute Endpoint | The actual running PostgreSQL instance that you connect to. It provides the CPU and RAM for processing queries. | The "server" or "engine" for your database. It can be started, suspended (scaled to zero), and resized. | Is attached to a single Branch. Your connection string points to a Compute Endpoint's hostname. |
| Database | A logical container for your data (tables, schemas, views) within a branch. It follows standard PostgreSQL conventions. | A single database within a PostgreSQL server instance. | Exists within a Branch. A branch can have multiple databases. |
| Role | A PostgreSQL role used for authentication (logging in) and authorization (permissions to access data). | A database user account with a username and password. | Belongs to a Branch. Roles from a parent branch are copied to child branches upon creation. |
| API Key | A secret token used to authenticate requests to the Neon API. Keys have different scopes (Personal, Organization, Project-scoped). | A password for programmatic access, allowing you to manage all other Neon resources. | Authenticates actions on Organizations, Projects, Branches, etc. |
| Operation | An asynchronous action performed by the Neon control plane, such as creating a branch or starting a compute. | A background job or task. Its status can be polled to know when an action is complete. | Associated with a Project and often a specific Branch or Endpoint. Essential for scripting API calls. |
### Understanding API key types
When performing actions via the API, you must select the correct type of API key based on the required scope and permissions. There are three types:
1. Personal API Key
- Scope: Accesses all projects that the user who created the key is a member of.
- Permissions: The key has the same permissions as its owner. If the user's access is revoked from an organization, the key loses access too.
- Best For: Individual use, scripting, and tasks tied to a specific user's permissions.
- Created By: Any user.
2. Organization API Key
- Scope: Accesses all projects and resources within an entire organization.
- Permissions: Has admin-level access across the organization, independent of any single user. It remains valid even if the creator leaves the organization.
- Best For: CI/CD pipelines, organization-wide automation, and service accounts that need broad access.
- Created By: Organization administrators only.
3. Project-scoped API Key
- Scope: Access is strictly limited to a single, specified project.
- Permissions: Cannot perform organization-level actions (like creating new projects) or delete the project it is scoped to. This is the most secure and limited key type.
- Best For: Project-specific integrations, third-party services, or automation that should be isolated to one project.
- Created By: Any organization member.
@@ -0,0 +1,92 @@
## Overview
This document outlines the rules for managing Neon API keys programmatically. It covers listing existing keys, creating new keys, and revoking keys.
### Important note on creating API keys
To create new API keys using the API, you must already possess a valid Personal API Key. The first key must be created from the Neon Console. You can ask the user to create one for you if you do not have one.
### List API keys
- Endpoint: `GET /api_keys`
- Authorization: Use a Personal API Key.
Example request:
```bash
curl "https://console.neon.tech/api/v2/api_keys" \
-H "Authorization: Bearer $PERSONAL_API_KEY"
```
Example response:
```json
[
{
"id": 2291506,
"name": "my-personal-key",
"created_at": "2025-09-10T09:44:04Z",
"created_by": {
"id": "487de658-08ba-4363-b387-86d18b9ad1c8",
"name": "<USER_NAME>",
"image": "<USER_IMAGE_URL>"
},
"last_used_at": "2025-09-10T09:44:09Z",
"last_used_from_addr": "49.43.218.132,34.211.200.85"
}
]
```
### Create an API key
- Endpoint: `POST /api_keys`
- Authorization: Use a Personal API Key.
- Body: Must include a `key_name`.
Example request:
```bash
curl https://console.neon.tech/api/v2/api_keys \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $PERSONAL_API_KEY" \
-d '{"key_name": "my-new-key"}'
```
Example response:
```json
{
"id": 2291515,
"key": "napi_9tlr13774gizljemrr133j5koy3bmsphj8iu38mh0yjl9q4r1b0jy2wuhhuxouzr",
"name": "my-new-key",
"created_at": "2025-09-10T09:47:59Z",
"created_by": "487de658-08ba-4363-b387-86d18b9ad1c8"
}
```
### Revoke an API key
- Endpoint: `DELETE /api_keys/{key_id}`
- Authorization: Use a Personal API Key.
Example request:
```bash
curl -X DELETE \
'https://console.neon.tech/api/v2/api_keys/2291515' \
-H "Authorization: Bearer $PERSONAL_API_KEY"
```
Example response:
```json
{
"id": 2291515,
"name": "mynewkey",
"created_at": "2025-09-10T09:47:59Z",
"created_by": "487de658-08ba-4363-b387-86d18b9ad1c8",
"last_used_at": "2025-09-10T09:53:01Z",
"last_used_from_addr": "2405:201:c01f:7013:d962:2b4f:2740:9750",
"revoked": true
}
```
@@ -0,0 +1,146 @@
## Overview
This document outlines the rules for managing and monitoring long-running operations in Neon, including branch creation and compute management.
## Operations
An operation is an action performed by the Neon Control Plane (e.g., `create_branch`, `start_compute`). When using the API programmatically, it is crucial to monitor the status of long-running operations to ensure one has completed before starting another that depends on it. Operations older than 6 months may be deleted from Neon's systems.
### List operations
1. Action: Retrieves a list of operations for the specified Neon project. The number of operations can be large, so pagination is recommended.
2. Endpoint: `GET /projects/{project_id}/operations`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project whose operations you want to list.
4. Query Parameters:
- `limit` (integer, optional): The number of operations to return in the response. Must be between 1 and 1000.
- `cursor` (string, optional): The cursor value from a previous response to fetch the next page of operations.
5. Procedure:
- Make an initial request with a `limit` to get the first page of results.
- The response will contain a `pagination.cursor` value.
- To get the next page, make a subsequent request including both the `limit` and the `cursor` from the previous response.
Example request
```bash
curl 'https://console.neon.tech/api/v2/projects/hidden-river-50598307/operations' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
Example response
```json
{
"operations": [
{
"id": "639f7f73-0b76-4749-a767-2d3c627ca5a6",
"project_id": "hidden-river-50598307",
"branch_id": "br-long-feather-adpbgzlx",
"endpoint_id": "ep-round-morning-adtpn2oc",
"action": "apply_config",
"status": "finished",
"failures_count": 0,
"created_at": "2025-09-10T12:15:23Z",
"updated_at": "2025-09-10T12:15:23Z",
"total_duration_ms": 87
},
{
"id": "b5a7882b-a5b3-4292-ad27-bffe733feae4",
"project_id": "hidden-river-50598307",
"branch_id": "br-super-wildflower-adniii9u",
"endpoint_id": "ep-ancient-brook-ad5ea04d",
"action": "apply_config",
"status": "finished",
"failures_count": 0,
"created_at": "2025-09-10T12:15:23Z",
"updated_at": "2025-09-10T12:15:23Z",
"total_duration_ms": 49
},
{
"id": "36a1cba0-97f1-476d-af53-d9e0d3a3606d",
"project_id": "hidden-river-50598307",
"branch_id": "br-super-wildflower-adniii9u",
"endpoint_id": "ep-ancient-brook-ad5ea04d",
"action": "start_compute",
"status": "finished",
"failures_count": 0,
"created_at": "2025-09-10T12:15:04Z",
"updated_at": "2025-09-10T12:15:05Z",
"total_duration_ms": 913
},
{
"id": "409c35ef-cbc3-4f1b-a4ca-f2de319f5360",
"project_id": "hidden-river-50598307",
"branch_id": "br-super-wildflower-adniii9u",
"action": "create_branch",
"status": "finished",
"failures_count": 0,
"created_at": "2025-09-10T12:15:04Z",
"updated_at": "2025-09-10T12:15:04Z",
"total_duration_ms": 136
},
{
"id": "274e240f-e2fb-4719-b796-c1ab7c4ae91c",
"project_id": "hidden-river-50598307",
"branch_id": "br-long-feather-adpbgzlx",
"endpoint_id": "ep-round-morning-adtpn2oc",
"action": "start_compute",
"status": "finished",
"failures_count": 0,
"created_at": "2025-09-10T12:14:58Z",
"updated_at": "2025-09-10T12:15:03Z",
"total_duration_ms": 4843
},
{
"id": "22ef6fbd-21c5-4cdb-9825-b0f9afddbb0d",
"project_id": "hidden-river-50598307",
"branch_id": "br-long-feather-adpbgzlx",
"action": "create_timeline",
"status": "finished",
"failures_count": 0,
"created_at": "2025-09-10T12:14:58Z",
"updated_at": "2025-09-10T12:15:01Z",
"total_duration_ms": 3096
}
],
"pagination": {
"cursor": "2025-09-10T12:14:58.848485Z"
}
}
```
### Retrieve operation details
1. Action: Retrieves the details and status of a single, specified operation. The `operation_id` is found in the response body of the initial API call that initiated it, or by listing operations.
2. Endpoint: `GET /projects/{project_id}/operations/{operation_id}`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project where the operation occurred.
- `operation_id` (UUID, required): The unique identifier of the operation. This ID is returned in the response body of the API call that initiated the operation.
Example request:
```bash
curl 'https://console.neon.tech/api/v2/projects/hidden-river-50598307/operations/274e240f-e2fb-4719-b796-c1ab7c4ae91c' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
Example response:
```json
{
"operation": {
"id": "274e240f-e2fb-4719-b796-c1ab7c4ae91c",
"project_id": "hidden-river-50598307",
"branch_id": "br-long-feather-adpbgzlx",
"endpoint_id": "ep-round-morning-adtpn2oc",
"action": "start_compute",
"status": "finished",
"failures_count": 0,
"created_at": "2025-09-10T12:14:58Z",
"updated_at": "2025-09-10T12:15:03Z",
"total_duration_ms": 4843
}
}
```
@@ -0,0 +1,199 @@
## Overview
This section provides rules for managing organizations, their members, invitations, and organization API keys. Organizations allow multiple users to collaborate on projects and share resources within Neon.
## Manage organizations
### Retrieve organization details
1. Action: Retrieves detailed information about a specific organization.
2. Endpoint: `GET /organizations/{org_id}`
3. Path Parameters:
- `org_id` (string, required): The unique identifier of the organization.
Example Request:
```bash
curl 'https://console.neon.tech/api/v2/organizations/{org_id}' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
### List organization members
1. Action: Retrieves a list of all members belonging to the specified organization.
2. Endpoint: `GET /organizations/{org_id}/members`
3. Path Parameters:
- `org_id` (string, required): The unique identifier of the organization.
Example Request:
```bash
curl 'https://console.neon.tech/api/v2/organizations/{org_id}/members' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
### Retrieve organization member details
1. Action: Retrieves information about a specific member of an organization.
2. Endpoint: `GET /organizations/{org_id}/members/{member_id}`
3. Path Parameters:
- `org_id` (string, required): The unique identifier of the organization.
- `member_id` (UUID, required): The unique identifier of the organization member.
Example Request:
```bash
curl 'https://console.neon.tech/api/v2/organizations/{org_id}/members/{member_id}' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
### Update role for organization member
1. Action: Updates the role of a specified member within an organization.
2. Prerequisite: This action can only be performed by an organization `admin`.
3. Endpoint: `PATCH /organizations/{org_id}/members/{member_id}`
4. Path Parameters:
- `org_id` (string, required): The unique identifier of the organization.
- `member_id` (UUID, required): The unique identifier of the organization member.
5. Body Parameters:
- `role` (string, required): The new role for the member. Allowed values: `admin`, `member`.
Example: Change a member's role to admin
```bash
curl -X 'PATCH' \
'https://console.neon.tech/api/v2/organizations/{org_id}/members/{member_id}' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY" \
-H 'Content-Type: application/json' \
-d '{"role": "admin"}'
```
### Remove member from organization
1. Action: Removes a specified member from an organization.
2. Prerequisites:
- This action can only be performed by an organization `admin`.
- An admin cannot be removed if they are the only admin left in the organization.
3. Endpoint: `DELETE /organizations/{org_id}/members/{member_id}`
4. Path Parameters:
- `org_id` (string, required): The unique identifier of the organization.
- `member_id` (UUID, required): The unique identifier of the organization member to remove.
Example Request:
```bash
curl -X 'DELETE' \
'https://console.neon.tech/api/v2/organizations/{org_id}/members/{member_id}' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
### Create organization invitations
1. Action: Creates and sends one or more email invitations for users to join a specific organization.
2. Endpoint: `POST /organizations/{org_id}/invitations`
3. Path Parameters:
- `org_id` (string, required): The unique identifier of the organization.
4. Body Parameters:
`invitations` (array of objects, required): A list of invitations to create.
- `email` (string, required): The email address of the user to invite.
- `role` (string, required): The role the invited user will have. Allowed values: `admin`, `member`.
Example: Invite two users with different roles
```bash
curl -X 'POST' \
'https://console.neon.tech/api/v2/organizations/{org_id}/invitations' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY" \
-H 'Content-Type: application/json' \
-d '{
"invitations": [
{
"email": "developer@example.com",
"role": "member"
},
{
"email": "manager@example.com",
"role": "admin"
}
]
}'
```
### List organization invitations
1. Action: Retrieves information about outstanding invitations for the specified organization.
2. Endpoint: `GET /organizations/{org_id}/invitations`
3. Path Parameters:
- `org_id` (string, required): The unique identifier of the organization.
Example Request:
```bash
curl 'https://console.neon.tech/api/v2/organizations/{org_id}/invitations' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
### Create organization API key
1. Action: Creates a new API key for the specified organization. The key can be scoped to the entire organization or limited to a single project within it.
2. Endpoint: `POST /organizations/{org_id}/api_keys`
3. Path Parameters:
- `org_id` (string, required): The unique identifier of the organization.
4. Body Parameters:
- `key_name` (string, required): A user-specified name for the API key (max 64 characters).
- `project_id` (string, optional): If provided, the API key's access will be restricted to only this project.
5. Authorization: Use a Personal API Key of an organization `admin` to create organization API keys.
Example: Create a project-scoped API key
```bash
curl -X 'POST' \
'https://console.neon.tech/api/v2/organizations/{org_id}/api_keys' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $PERSONAL_API_KEY_OF_ADMIN" \
-H 'Content-Type: application/json' \
-d '{
"key_name": "ci-pipeline-key-for-project-x",
"project_id": "project-id-123"
}'
```
### List organization API keys
1. Action: Retrieves a list of all API keys created for the specified organization.
2. Endpoint: `GET /organizations/{org_id}/api_keys`
3. Note: The response includes metadata about the keys (like `id` and `name`) but does not include the secret key tokens themselves. Tokens are only visible upon creation.
4. Path Parameters:
- `org_id` (string, required): The unique identifier of the organization.
Example Request:
```bash
curl 'https://console.neon.tech/api/v2/organizations/{org_id}/api_keys' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
### Revoke organization API key
1. Action: Permanently revokes the specified organization API key.
2. Endpoint: `DELETE /organizations/{org_id}/api_keys/{key_id}`
3. Path Parameters:
- `org_id` (string, required): The unique identifier of the organization.
- `key_id` (integer, required): The unique identifier of the API key to revoke. You can obtain this ID by listing the organization's API keys.
Example Request:
```bash
curl -X 'DELETE' \
'https://console.neon.tech/api/v2/organizations/{org_id}/api_keys/{key_id}' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
@@ -0,0 +1,576 @@
## Overview
This document outlines the rules for managing Neon projects programmatically. It covers creation, retrieval, updates, and deletion.
## Manage projects
### List projects
1. Action: Retrieves a list of all projects accessible to the account associated with the API key. This is the primary method for obtaining `project_id` values required for other API calls.
2. Endpoint: `GET /projects`
3. Query Parameters:
- `limit` (optional, integer, default: 10): Specifies the number of projects to return, from 1 to 400.
- `cursor` (optional, string): Used for pagination. Provide the `cursor` value from a previous response to fetch the next set of projects.
- `search` (optional, string): Filters projects by a partial match on the project `name` or `id`.
- `org_id` (optional, string): Filters projects by a specific organization ID.
4. When iterating through all projects, use a combination of the `limit` and `cursor` parameters to handle pagination correctly.
Example request:
```bash
# Retrieve the first 10 projects
curl 'https://console.neon.tech/api/v2/projects?limit=10' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
Example response:
```json
{
"projects": [
{
"id": "old-fire-32990194",
"platform_id": "aws",
"region_id": "aws-ap-southeast-1",
"name": "old-fire-32990194",
"provisioner": "k8s-neonvm",
"default_endpoint_settings": {
"autoscaling_limit_min_cu": 0.25,
"autoscaling_limit_max_cu": 2,
"suspend_timeout_seconds": 0
},
"settings": {
"allowed_ips": {
"ips": [],
"protected_branches_only": false
},
"enable_logical_replication": false,
"maintenance_window": {
"weekdays": [5],
"start_time": "19:00",
"end_time": "20:00"
},
"block_public_connections": false,
"block_vpc_connections": false,
"hipaa": false
},
"pg_version": 17,
"proxy_host": "ap-southeast-1.aws.neon.tech",
"branch_logical_size_limit": 512,
"branch_logical_size_limit_bytes": 536870912,
"store_passwords": true,
"active_time": 0,
"cpu_used_sec": 0,
"creation_source": "console",
"created_at": "2025-09-10T06:58:33Z",
"updated_at": "2025-09-10T06:58:39Z",
"synthetic_storage_size": 0,
"quota_reset_at": "2025-10-01T00:00:00Z",
"owner_id": "org-royal-sun-91776391",
"compute_last_active_at": "2025-09-10T06:58:38Z",
"org_id": "org-royal-sun-91776391",
"history_retention_seconds": 86400
}
],
"pagination": {
"cursor": "old-fire-32990194"
},
"applications": {},
"integrations": {}
}
```
### Create project
1. Action: Creates a new Neon project. You can specify a wide range of settings at creation time, including the region, Postgres version, default branch and compute configurations, and security settings.
2. Endpoint: `POST /projects`
3. Body Parameters: The request body must contain a top-level `project` object with the following nested attributes:
`project` (object, required): The main container for all project settings.
- `name` (string, optional): A descriptive name for the project (1-256 characters). If omitted, the project name will be identical to its generated ID.
- `pg_version` (integer, optional): The major Postgres version. Defaults to `17`. Supported versions: 14, 15, 16, 17, 18.
- `region_id` (string, optional): The identifier for the region where the project will be created (e.g., `aws-us-east-1`).
- `org_id` (string, optional): The ID of an organization to which the project will belong. Required if using an Organization API key.
- `store_passwords` (boolean, optional): Whether to store role passwords in Neon. Storing passwords is required for features like the SQL Editor and integrations.
- `history_retention_seconds` (integer, optional): The duration in seconds (0 to 2,592,000) to retain project history for features like Point-in-Time Restore. Defaults to 86400 (1 day).
- `provisioner` (string, optional): The compute provisioner. Specify `k8s-neonvm` to enable Autoscaling. Allowed values: `k8s-pod`, `k8s-neonvm`.
- `default_endpoint_settings` (object, optional): Default settings for new compute endpoints created in this project.
- `autoscaling_limit_min_cu` (number, optional): The minimum number of Compute Units (CU). Minimum value is `0.25`.
- `autoscaling_limit_max_cu` (number, optional): The maximum number of Compute Units (CU). Minimum value is `0.25`.
- `suspend_timeout_seconds` (integer, optional): Duration of inactivity in seconds before a compute is suspended. Ranges from -1 (never suspend) to 604800 (1 week). A value of `0` uses the default of 300 seconds (5 minutes).
- `settings` (object, optional): Project-wide settings.
- `quota` (object, optional): Per-project consumption quotas. A zero or empty value means "unlimited".
- `active_time_seconds` (integer, optional): Wall-clock time allowance for active computes.
- `compute_time_seconds` (integer, optional): CPU seconds allowance.
- `written_data_bytes` (integer, optional): Data written allowance.
- `data_transfer_bytes` (integer, optional): Data transferred allowance.
- `logical_size_bytes` (integer, optional): Logical data size limit per branch.
- `allowed_ips` (object, optional): Configures the IP Allowlist.
- `ips` (array of strings, optional): A list of allowed IP addresses or CIDR ranges.
- `protected_branches_only` (boolean, optional): If `true`, the IP allowlist applies only to protected branches.
- `enable_logical_replication` (boolean, optional): Sets `wal_level=logical`.
- `maintenance_window` (object, optional): The time period for scheduled maintenance.
- `weekdays` (array of integers, required if `maintenance_window` is set): Days of the week (1=Monday, 7=Sunday).
- `start_time` (string, required if `maintenance_window` is set): Start time in "HH:MM" UTC format.
- `end_time` (string, required if `maintenance_window` is set): End time in "HH:MM" UTC format.
- `branch` (object, optional): Configuration for the project's default branch.
- `name` (string, optional): The name for the default branch. Defaults to `main`.
- `role_name` (string, optional): The name for the default role. Defaults to `{database_name}_owner`.
- `database_name` (string, optional): The name for the default database. Defaults to `neondb`.
Example request
```bash
curl -X POST 'https://console.neon.tech/api/v2/projects' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY" \
-H 'Content-Type: application/json' \
-d '{
"project": {
"name": "my-new-api-project",
"pg_version": 17
}
}'
```
Example response
```json
{
"project": {
"data_storage_bytes_hour": 0,
"data_transfer_bytes": 0,
"written_data_bytes": 0,
"compute_time_seconds": 0,
"active_time_seconds": 0,
"cpu_used_sec": 0,
"id": "sparkling-hill-99143322",
"platform_id": "aws",
"region_id": "aws-us-west-2",
"name": "my-new-api-project",
"provisioner": "k8s-neonvm",
"default_endpoint_settings": {
"autoscaling_limit_min_cu": 0.25,
"autoscaling_limit_max_cu": 0.25,
"suspend_timeout_seconds": 0
},
"settings": {
"allowed_ips": {
"ips": [],
"protected_branches_only": false
},
"enable_logical_replication": false,
"maintenance_window": {
"weekdays": [5],
"start_time": "07:00",
"end_time": "08:00"
},
"block_public_connections": false,
"block_vpc_connections": false,
"hipaa": false
},
"pg_version": 17,
"proxy_host": "c-2.us-west-2.aws.neon.tech",
"branch_logical_size_limit": 512,
"branch_logical_size_limit_bytes": 536870912,
"store_passwords": true,
"creation_source": "console",
"history_retention_seconds": 86400,
"created_at": "2025-09-10T07:58:16Z",
"updated_at": "2025-09-10T07:58:16Z",
"consumption_period_start": "0001-01-01T00:00:00Z",
"consumption_period_end": "0001-01-01T00:00:00Z",
"owner_id": "org-royal-sun-91776391",
"org_id": "org-royal-sun-91776391"
},
"connection_uris": [
{
"connection_uri": "postgresql://neondb_owner:npg_N67FDMtGvJke@ep-round-unit-afbn7qv4.c-2.us-west-2.aws.neon.tech/neondb?sslmode=require",
"connection_parameters": {
"database": "neondb",
"password": "npg_N67FDMtGvJke",
"role": "neondb_owner",
"host": "ep-round-unit-afbn7qv4.c-2.us-west-2.aws.neon.tech",
"pooler_host": "ep-round-unit-afbn7qv4-pooler.c-2.us-west-2.aws.neon.tech"
}
}
],
"roles": [
{
"branch_id": "br-green-mode-afe3fl9y",
"name": "neondb_owner",
"password": "npg_N67FDMtGvJke",
"protected": false,
"created_at": "2025-09-10T07:58:16Z",
"updated_at": "2025-09-10T07:58:16Z"
}
],
"databases": [
{
"id": 6677853,
"branch_id": "br-green-mode-afe3fl9y",
"name": "neondb",
"owner_name": "neondb_owner",
"created_at": "2025-09-10T07:58:16Z",
"updated_at": "2025-09-10T07:58:16Z"
}
],
"operations": [
{
"id": "08b9367d-6918-4cd5-b4a6-41c8fd984b7e",
"project_id": "sparkling-hill-99143322",
"branch_id": "br-green-mode-afe3fl9y",
"action": "create_timeline",
"status": "running",
"failures_count": 0,
"created_at": "2025-09-10T07:58:16Z",
"updated_at": "2025-09-10T07:58:16Z",
"total_duration_ms": 0
},
{
"id": "c6917f04-5cd3-48a2-97c9-186b1d9729f0",
"project_id": "sparkling-hill-99143322",
"branch_id": "br-green-mode-afe3fl9y",
"endpoint_id": "ep-round-unit-afbn7qv4",
"action": "start_compute",
"status": "scheduling",
"failures_count": 0,
"created_at": "2025-09-10T07:58:16Z",
"updated_at": "2025-09-10T07:58:16Z",
"total_duration_ms": 0
}
],
"branch": {
"id": "br-green-mode-afe3fl9y",
"project_id": "sparkling-hill-99143322",
"name": "main",
"current_state": "init",
"pending_state": "ready",
"state_changed_at": "2025-09-10T07:58:16Z",
"creation_source": "console",
"primary": true,
"default": true,
"protected": false,
"cpu_used_sec": 0,
"compute_time_seconds": 0,
"active_time_seconds": 0,
"written_data_bytes": 0,
"data_transfer_bytes": 0,
"created_at": "2025-09-10T07:58:16Z",
"updated_at": "2025-09-10T07:58:16Z",
"init_source": "parent-data"
},
"endpoints": [
{
"host": "ep-round-unit-afbn7qv4.c-2.us-west-2.aws.neon.tech",
"id": "ep-round-unit-afbn7qv4",
"project_id": "sparkling-hill-99143322",
"branch_id": "br-green-mode-afe3fl9y",
"autoscaling_limit_min_cu": 0.25,
"autoscaling_limit_max_cu": 0.25,
"region_id": "aws-us-west-2",
"type": "read_write",
"current_state": "init",
"pending_state": "active",
"settings": {},
"pooler_enabled": false,
"pooler_mode": "transaction",
"disabled": false,
"passwordless_access": true,
"creation_source": "console",
"created_at": "2025-09-10T07:58:16Z",
"updated_at": "2025-09-10T07:58:16Z",
"proxy_host": "c-2.us-west-2.aws.neon.tech",
"suspend_timeout_seconds": 0,
"provisioner": "k8s-neonvm"
}
]
}
```
### Retrieve project details
1. Action: Retrieves detailed information about a single, specific project.
2. Endpoint: `GET /projects/{project_id}`
3. Prerequisite: You must have the `project_id` of the project you wish to retrieve.
4. Path Parameters:
- `project_id` (required, string): The unique identifier of the project.
Example request:
```bash
curl 'https://console.neon.tech/api/v2/projects/sparkling-hill-99143322' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
Example response
```json
{
"project": {
"data_storage_bytes_hour": 0,
"data_transfer_bytes": 0,
"written_data_bytes": 0,
"compute_time_seconds": 0,
"active_time_seconds": 0,
"cpu_used_sec": 0,
"id": "sparkling-hill-99143322",
"platform_id": "aws",
"region_id": "aws-us-west-2",
"name": "my-new-api-project",
"provisioner": "k8s-neonvm",
"default_endpoint_settings": {
"autoscaling_limit_min_cu": 0.25,
"autoscaling_limit_max_cu": 0.25,
"suspend_timeout_seconds": 0
},
"settings": {
"allowed_ips": {
"ips": [],
"protected_branches_only": false
},
"enable_logical_replication": false,
"maintenance_window": {
"weekdays": [5],
"start_time": "07:00",
"end_time": "08:00"
},
"block_public_connections": false,
"block_vpc_connections": false,
"hipaa": false
},
"pg_version": 17,
"proxy_host": "c-2.us-west-2.aws.neon.tech",
"branch_logical_size_limit": 512,
"branch_logical_size_limit_bytes": 536870912,
"store_passwords": true,
"creation_source": "console",
"history_retention_seconds": 86400,
"created_at": "2025-09-10T07:58:16Z",
"updated_at": "2025-09-10T07:58:25Z",
"synthetic_storage_size": 0,
"consumption_period_start": "2025-09-10T06:58:15Z",
"consumption_period_end": "2025-10-01T00:00:00Z",
"owner_id": "org-royal-sun-91776391",
"owner": {
"email": "<USER_EMAIL>",
"name": "My Personal Account",
"branches_limit": 10,
"subscription_type": "free_v3"
},
"compute_last_active_at": "2025-09-10T07:58:21Z",
"org_id": "org-royal-sun-91776391"
}
}
```
### Update a project
1. Action: Updates the settings of a specified project. This endpoint is used to modify a wide range of project attributes after creation, such as its name, default compute settings, security policies, and maintenance schedules.
2. Endpoint: `PATCH /projects/{project_id}`
3. Path Parameters:
- `project_id` (string, required): The unique identifier of the project to update.
4. Body Parameters: The request body must contain a top-level `project` object with the attributes to be updated.
`project` (object, required): The main container for the settings you want to modify.
- `name` (string, optional): A new descriptive name for the project.
- `history_retention_seconds` (integer, optional): The duration in seconds (0 to 2,592,000) to retain project history.
- `default_endpoint_settings` (object, optional): New default settings for compute endpoints created in this project.
- `autoscaling_limit_min_cu` (number, optional): The minimum number of Compute Units (CU). Minimum `0.25`.
- `autoscaling_limit_max_cu` (number, optional): The maximum number of Compute Units (CU). Minimum `0.25`.
- `suspend_timeout_seconds` (integer, optional): Duration of inactivity in seconds before a compute is suspended. Ranges from -1 (never suspend) to 604800 (1 week). A value of `0` uses the default of 300 seconds (5 minutes).
- `settings` (object, optional): Project-wide settings to update.
- `quota` (object, optional): Per-project consumption quotas.
- `active_time_seconds` (integer, optional): Wall-clock time allowance for active computes.
- `compute_time_seconds` (integer, optional): CPU seconds allowance.
- `written_data_bytes` (integer, optional): Data written allowance.
- `data_transfer_bytes` (integer, optional): Data transferred allowance.
- `logical_size_bytes` (integer, optional): Logical data size limit per branch.
- `allowed_ips` (object, optional): Modifies the IP Allowlist.
- `ips` (array of strings, optional): The new list of allowed IP addresses or CIDR ranges.
- `protected_branches_only` (boolean, optional): If `true`, the IP allowlist applies only to protected branches.
- `enable_logical_replication` (boolean, optional): Sets `wal_level=logical`. This is irreversible.
- `maintenance_window` (object, optional): The time period for scheduled maintenance.
- `weekdays` (array of integers, required if `maintenance_window` is set): Days of the week (1=Monday, 7=Sunday).
- `start_time` (string, required if `maintenance_window` is set): Start time in "HH:MM" UTC format.
- `end_time` (string, required if `maintenance_window` is set): End time in "HH:MM" UTC format.
- `block_public_connections` (boolean, optional): If `true`, disallows connections from the public internet.
- `block_vpc_connections` (boolean, optional): If `true`, disallows connections from VPC endpoints.
- `audit_log_level` (string, optional): Sets the audit log level. Allowed values: `base`, `extended`, `full`.
- `hipaa` (boolean, optional): Toggles HIPAA compliance settings.
- `preload_libraries` (object, optional): Libraries to preload into compute instances.
- `use_defaults` (boolean, optional): Toggles the use of default libraries.
- `enabled_libraries` (array of strings, optional): A list of specific libraries to enable.
Example request
```bash
curl -X PATCH 'https://console.neon.tech/api/v2/projects/sparkling-hill-99143322' \
-H 'accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY" \
-H 'Content-Type: application/json' \
-d '{
"project": {
"name": "updated-project-name"
}
}'
```
Example response
```json
{
"project": {
"data_storage_bytes_hour": 0,
"data_transfer_bytes": 0,
"written_data_bytes": 29060360,
"compute_time_seconds": 79,
"active_time_seconds": 308,
"cpu_used_sec": 79,
"id": "sparkling-hill-99143322",
"platform_id": "aws",
"region_id": "aws-us-west-2",
"name": "updated-project-name",
"provisioner": "k8s-neonvm",
"default_endpoint_settings": {
"autoscaling_limit_min_cu": 0.25,
"autoscaling_limit_max_cu": 0.25,
"suspend_timeout_seconds": 0
},
"settings": {
"allowed_ips": {
"ips": [],
"protected_branches_only": false
},
"enable_logical_replication": false,
"maintenance_window": {
"weekdays": [5],
"start_time": "07:00",
"end_time": "08:00"
},
"block_public_connections": false,
"block_vpc_connections": false,
"hipaa": false
},
"pg_version": 17,
"proxy_host": "c-2.us-west-2.aws.neon.tech",
"branch_logical_size_limit": 512,
"branch_logical_size_limit_bytes": 536870912,
"store_passwords": true,
"creation_source": "console",
"history_retention_seconds": 86400,
"created_at": "2025-09-10T07:58:16Z",
"updated_at": "2025-09-10T08:08:23Z",
"synthetic_storage_size": 0,
"consumption_period_start": "0001-01-01T00:00:00Z",
"consumption_period_end": "0001-01-01T00:00:00Z",
"owner_id": "org-royal-sun-91776391",
"compute_last_active_at": "2025-09-10T07:58:21Z"
},
"operations": []
}
```
### Delete project
1. Action: Permanently deletes a project and all of its associated resources, including all branches, computes, databases, and roles.
2. Endpoint: `DELETE /projects/{project_id}`
3. Prerequisite: You must have the `project_id` of the project you wish to delete.
4. Warning: This is a destructive action that cannot be undone. It deletes all data, databases, and resources in the project. Proceed with extreme caution and confirm with the user before executing this operation.
5. Path Parameters:
- `project_id` (required, string): The unique identifier of the project to be deleted.
Example request:
```bash
curl -X 'DELETE' \
'https://console.neon.tech/api/v2/projects/sparkling-hill-99143322' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
Example response:
```json
{
"project": {
"data_storage_bytes_hour": 0,
"data_transfer_bytes": 0,
"written_data_bytes": 29060360,
"compute_time_seconds": 79,
"active_time_seconds": 308,
"cpu_used_sec": 79,
"id": "sparkling-hill-99143322",
"platform_id": "aws",
"region_id": "aws-us-west-2",
"name": "updated-project-name",
"provisioner": "k8s-neonvm",
"default_endpoint_settings": {
"autoscaling_limit_min_cu": 0.25,
"autoscaling_limit_max_cu": 0.25,
"suspend_timeout_seconds": 0
},
"settings": {
"allowed_ips": {
"ips": [],
"protected_branches_only": false
},
"enable_logical_replication": false,
"maintenance_window": {
"weekdays": [5],
"start_time": "07:00",
"end_time": "08:00"
},
"block_public_connections": false,
"block_vpc_connections": false,
"hipaa": false
},
"pg_version": 17,
"proxy_host": "c-2.us-west-2.aws.neon.tech",
"branch_logical_size_limit": 512,
"branch_logical_size_limit_bytes": 536870912,
"store_passwords": true,
"creation_source": "console",
"history_retention_seconds": 86400,
"created_at": "2025-09-10T07:58:16Z",
"updated_at": "2025-09-10T08:08:23Z",
"synthetic_storage_size": 0,
"consumption_period_start": "0001-01-01T00:00:00Z",
"consumption_period_end": "0001-01-01T00:00:00Z",
"owner_id": "org-royal-sun-91776391",
"compute_last_active_at": "2025-09-10T07:58:21Z",
"org_id": "org-royal-sun-91776391"
}
}
```
### Retrieve connection URI
1. Action: Retrieves a ready-to-use connection URI for a specific database within a project.
2. Endpoint: `GET /projects/{project_id}/connection_uri`
3. Prerequisites: You must know the `project_id`, `database_name`, and `role_name`.
4. Query Parameters:
- `project_id` (path, required): The unique identifier of the project.
- `database_name` (query, required): The name of the target database.
- `role_name` (query, required): The role to use for the connection.
- `branch_id` (query, optional): The branch ID. Defaults to the project's primary branch if not specified.
- `pooled` (query, optional, boolean): If set to `false`, returns a direct connection URI instead of a pooled one. Defaults to `true`.
- `endpoint_id` (query, optional): The specific endpoint ID to connect to. Defaults to the `read-write` endpoint_id associated with the `branch_id` if not specified.
Example request:
```bash
curl 'https://console.neon.tech/api/v2/projects/old-fire-32990194/connection_uri?database_name=neondb&role_name=neondb_owner' \
-H 'Accept: application/json' \
-H "Authorization: Bearer $NEON_API_KEY"
```
Example response:
```json
{
"uri": "postgresql://neondb_owner:npg_IDNnorOST71P@ep-shiny-morning-a1bfdvjs-pooler.ap-southeast-1.aws.neon.tech/neondb?channel_binding=require&sslmode=require"
}
```
@@ -0,0 +1,254 @@
# Neon Serverless Driver
Patterns and best practices for connecting to Neon databases in serverless environments using the `@neondatabase/serverless` driver. The driver connects over **HTTP** for fast, single queries or **WebSockets** for `node-postgres` compatibility and interactive transactions.
For official documentation:
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/serverless/serverless-driver
```
## Installation
```bash
# Using npm
npm install @neondatabase/serverless
# Using JSR
bunx jsr add @neon/serverless
```
**Note:** Version 1.0.0+ requires **Node.js v19 or later**.
For projects that depend on `pg` but want to use Neon's WebSocket-based connection pool:
```json
"dependencies": {
"pg": "npm:@neondatabase/serverless@^0.10.4"
},
"overrides": {
"pg": "npm:@neondatabase/serverless@^0.10.4"
}
```
## Connection String
Always use environment variables:
```typescript
// For HTTP queries
import { neon } from "@neondatabase/serverless";
const sql = neon(process.env.DATABASE_URL!);
// For WebSocket connections
import { Pool } from "@neondatabase/serverless";
const pool = new Pool({ connectionString: process.env.DATABASE_URL! });
```
**Never hardcode credentials:**
```typescript
// AVOID
const sql = neon("postgres://username:password@host.neon.tech/neondb");
```
## HTTP Queries with `neon` function
Ideal for simple, "one-shot" queries in serverless/edge environments. Uses HTTP `fetch` - fastest method for single queries.
### Parameterized Queries
Use tagged template literals for safe parameter interpolation:
```typescript
const [post] = await sql`SELECT * FROM posts WHERE id = ${postId}`;
```
For manually constructed queries:
```typescript
const [post] = await sql.query("SELECT * FROM posts WHERE id = $1", [postId]);
```
**Never concatenate user input:**
```typescript
// AVOID: SQL Injection Risk
const [post] = await sql("SELECT * FROM posts WHERE id = " + postId);
```
### Configuration Options
```typescript
// Return rows as arrays instead of objects
const sqlArrayMode = neon(process.env.DATABASE_URL!, { arrayMode: true });
const rows = await sqlArrayMode`SELECT id, title FROM posts`;
// rows -> [[1, "First Post"], [2, "Second Post"]]
// Get full results including row count and field metadata
const sqlFull = neon(process.env.DATABASE_URL!, { fullResults: true });
const result = await sqlFull`SELECT * FROM posts LIMIT 1`;
// result -> { rows: [...], fields: [...], rowCount: 1, ... }
```
## WebSocket Connections with `Pool` and `Client`
Use for `node-postgres` compatibility, interactive transactions, or session support.
### WebSocket Configuration
For Node.js v21 and earlier:
```typescript
import { Pool, neonConfig } from "@neondatabase/serverless";
import ws from "ws";
// Required for Node.js < v22
neonConfig.webSocketConstructor = ws;
const pool = new Pool({ connectionString: process.env.DATABASE_URL! });
```
### Serverless Lifecycle Management
Create, use, and close the pool within the same invocation:
```typescript
// Vercel Edge Functions example
export default async (req: Request, ctx: ExecutionContext) => {
const pool = new Pool({ connectionString: process.env.DATABASE_URL! });
try {
const { rows } = await pool.query("SELECT * FROM users");
return new Response(JSON.stringify(rows));
} catch (err) {
console.error(err);
return new Response("Database error", { status: 500 });
} finally {
ctx.waitUntil(pool.end());
}
};
```
**Avoid** creating a global `Pool` instance outside the handler.
## Transactions
### HTTP Transactions
For running multiple queries in a single, non-interactive transaction:
```typescript
const [newUser, newProfile] = await sql.transaction(
[
sql`INSERT INTO users(name) VALUES(${name}) RETURNING id`,
sql`INSERT INTO profiles(user_id, bio) VALUES(${userId}, ${bio})`,
],
{
isolationLevel: "ReadCommitted",
readOnly: false,
},
);
```
### Interactive Transactions
For complex transactions with conditional logic:
```typescript
const pool = new Pool({ connectionString: process.env.DATABASE_URL! });
const client = await pool.connect();
try {
await client.query("BEGIN");
const {
rows: [{ id }],
} = await client.query("INSERT INTO users(name) VALUES($1) RETURNING id", [
name,
]);
await client.query("INSERT INTO profiles(user_id, bio) VALUES($1, $2)", [
id,
bio,
]);
await client.query("COMMIT");
} catch (err) {
await client.query("ROLLBACK");
throw err;
} finally {
client.release();
await pool.end();
}
```
## Environment-Specific Optimizations
```javascript
// For Vercel Edge Functions, specify nearest region
export const config = {
runtime: "edge",
regions: ["iad1"], // Region nearest to your Neon DB
};
// For Cloudflare Workers, consider using Hyperdrive
// https://neon.tech/blog/hyperdrive-neon-faq
```
## ORM Integration
For Drizzle ORM integration with the serverless driver, see `neon-drizzle.md`.
### Prisma
```typescript
import { neonConfig } from "@neondatabase/serverless";
import { PrismaNeon, PrismaNeonHTTP } from "@prisma/adapter-neon";
import { PrismaClient } from "@prisma/client";
import ws from "ws";
const connectionString = process.env.DATABASE_URL;
neonConfig.webSocketConstructor = ws;
// HTTP adapter
const adapterHttp = new PrismaNeonHTTP(connectionString!, {});
export const prismaClientHttp = new PrismaClient({ adapter: adapterHttp });
// WebSocket adapter
const adapterWs = new PrismaNeon({ connectionString });
export const prismaClientWs = new PrismaClient({ adapter: adapterWs });
```
### Kysely
```typescript
import { Pool } from "@neondatabase/serverless";
import { Kysely, PostgresDialect } from "kysely";
const dialect = new PostgresDialect({
pool: new Pool({ connectionString: process.env.DATABASE_URL }),
});
const db = new Kysely({ dialect });
```
**NOTE:** Do not pass the `neon()` function to ORMs that expect a `node-postgres` compatible `Pool`.
## Error Handling
```javascript
// Pool error handling
const pool = new Pool({ connectionString: process.env.DATABASE_URL });
pool.on("error", (err) => {
console.error("Unexpected error on idle client", err);
process.exit(-1);
});
// Query error handling
try {
const [post] = await sql`SELECT * FROM posts WHERE id = ${postId}`;
if (!post) {
return new Response("Not found", { status: 404 });
}
} catch (err) {
console.error("Database query failed:", err);
return new Response("Server error", { status: 500 });
}
```
@@ -0,0 +1,334 @@
# Neon TypeScript SDK
The `@neondatabase/api-client` TypeScript SDK is a typed wrapper around the Neon REST API. It provides methods for managing all Neon resources, including projects, branches, endpoints, roles, and databases.
For core concepts (Organization, Project, Branch, Endpoint, etc.), see `what-is-neon.md`.
## Documentation
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/reference/typescript-sdk
```
## Installation
```bash
npm install @neondatabase/api-client
```
## Authentication
```typescript
import { createApiClient } from "@neondatabase/api-client";
const apiKey = process.env.NEON_API_KEY;
if (!apiKey) {
throw new Error("NEON_API_KEY environment variable is not set.");
}
const apiClient = createApiClient({ apiKey });
```
## Projects
### List Projects
```typescript
const response = await apiClient.listProjects({});
console.log("Projects:", response.data.projects);
```
### Create Project
```typescript
const response = await apiClient.createProject({
project: { name: "my-project", pg_version: 17, region_id: "aws-us-east-2" },
});
console.log(
"Connection URI:",
response.data.connection_uris[0]?.connection_uri,
);
```
### Get Project
```typescript
const response = await apiClient.getProject("your-project-id");
```
### Update Project
```typescript
await apiClient.updateProject("your-project-id", {
project: { name: "new-name" },
});
```
### Delete Project
```typescript
await apiClient.deleteProject("project-id");
```
### Get Connection URI
```typescript
const response = await apiClient.getConnectionUri({
projectId: "your-project-id",
database_name: "neondb",
role_name: "neondb_owner",
pooled: true,
});
console.log("URI:", response.data.uri);
```
## Branches
### Create Branch
```typescript
import { EndpointType } from "@neondatabase/api-client";
const response = await apiClient.createProjectBranch("your-project-id", {
branch: { name: "feature-branch" },
endpoints: [{ type: EndpointType.ReadWrite, autoscaling_limit_max_cu: 1 }],
});
```
### List Branches
```typescript
const response = await apiClient.listProjectBranches({
projectId: "your-project-id",
});
```
### Get Branch
```typescript
const response = await apiClient.getProjectBranch("your-project-id", "br-xxx");
```
### Update Branch
```typescript
await apiClient.updateProjectBranch("your-project-id", "br-xxx", {
branch: { name: "updated-name" },
});
```
### Delete Branch
```typescript
await apiClient.deleteProjectBranch("your-project-id", "br-xxx");
```
## Databases
### Create Database
```typescript
await apiClient.createProjectBranchDatabase("your-project-id", "br-xxx", {
database: { name: "my-app-db", owner_name: "neondb_owner" },
});
```
### List Databases
```typescript
const response = await apiClient.listProjectBranchDatabases(
"your-project-id",
"br-xxx",
);
```
### Delete Database
```typescript
await apiClient.deleteProjectBranchDatabase(
"your-project-id",
"br-xxx",
"my-app-db",
);
```
## Roles
### Create Role
```typescript
const response = await apiClient.createProjectBranchRole(
"your-project-id",
"br-xxx",
{
role: { name: "app_user" },
},
);
console.log("Password:", response.data.role.password);
```
### List Roles
```typescript
const response = await apiClient.listProjectBranchRoles(
"your-project-id",
"br-xxx",
);
```
### Delete Role
```typescript
await apiClient.deleteProjectBranchRole(
"your-project-id",
"br-xxx",
"app_user",
);
```
## Endpoints
### Create Endpoint
```typescript
import { EndpointType } from "@neondatabase/api-client";
const response = await apiClient.createProjectEndpoint("your-project-id", {
endpoint: { branch_id: "br-xxx", type: EndpointType.ReadOnly },
});
```
### List Endpoints
```typescript
const response = await apiClient.listProjectEndpoints("your-project-id");
```
### Start/Suspend/Restart Endpoint
```typescript
// Start
await apiClient.startProjectEndpoint("your-project-id", "ep-xxx");
// Suspend
await apiClient.suspendProjectEndpoint("your-project-id", "ep-xxx");
// Restart
await apiClient.restartProjectEndpoint("your-project-id", "ep-xxx");
```
### Update Endpoint
```typescript
await apiClient.updateProjectEndpoint("your-project-id", "ep-xxx", {
endpoint: { autoscaling_limit_max_cu: 2 },
});
```
### Delete Endpoint
```typescript
await apiClient.deleteProjectEndpoint("your-project-id", "ep-xxx");
```
## API Keys
### List API Keys
```typescript
const response = await apiClient.listApiKeys();
```
### Create API Key
```typescript
const response = await apiClient.createApiKey({ key_name: "my-script-key" });
console.log("Key:", response.data.key); // Store securely!
```
### Revoke API Key
```typescript
await apiClient.revokeApiKey(1234);
```
## Operations
### List Operations
```typescript
const response = await apiClient.listProjectOperations({
projectId: "your-project-id",
});
```
### Get Operation
```typescript
const response = await apiClient.getProjectOperation(
"your-project-id",
"op-xxx",
);
```
## Organizations
### Get Organization
```typescript
const response = await apiClient.getOrganization("org-xxx");
```
### List Members
```typescript
const response = await apiClient.getOrganizationMembers("org-xxx");
```
### Create Org API Key
```typescript
const response = await apiClient.createOrgApiKey("org-xxx", {
key_name: "ci-key",
project_id: "project-xxx", // Optional: scope to project
});
```
### Invite Member
```typescript
import { MemberRole } from "@neondatabase/api-client";
await apiClient.createOrganizationInvitations("org-xxx", {
invitations: [{ email: "dev@example.com", role: MemberRole.Member }],
});
```
## Error Handling
```typescript
async function safeApiOperation(projectId: string) {
try {
const response = await apiClient.getProject(projectId);
return response.data;
} catch (error: any) {
if (error.isAxiosError) {
const status = error.response?.status;
switch (status) {
case 401:
console.error("Check your NEON_API_KEY");
break;
case 404:
console.error("Resource not found");
break;
case 429:
console.error("Rate limit exceeded");
break;
default:
console.error("API error:", error.response?.data?.message);
}
}
return null;
}
}
```
@@ -0,0 +1,78 @@
# Referencing Neon Docs
The Neon documentation is the source of truth for all Neon-related information. Always verify Neon-related claims, configurations, and best practices against the official documentation.
## Getting the Documentation Index
To get a list of all available Neon documentation pages:
```bash
curl https://neon.tech/llms.txt
```
This returns an index of all documentation pages with their URLs and descriptions.
## Fetching Individual Documentation Pages
To fetch any documentation page as markdown for review:
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/<path>
```
**Examples:**
```bash
# Fetch the API reference
curl -H "Accept: text/markdown" https://neon.tech/docs/reference/api-reference
# Fetch connection pooling docs
curl -H "Accept: text/markdown" https://neon.tech/docs/connect/connection-pooling
# Fetch branching documentation
curl -H "Accept: text/markdown" https://neon.tech/docs/introduction/branching
# Fetch serverless driver docs
curl -H "Accept: text/markdown" https://neon.tech/docs/serverless/serverless-driver
```
## Common Documentation Paths
| Topic | Path |
| ------------------ | ------------------------------------ |
| Introduction | `/docs/introduction` |
| Branching | `/docs/introduction/branching` |
| Autoscaling | `/docs/introduction/autoscaling` |
| Scale to Zero | `/docs/introduction/scale-to-zero` |
| Connection Pooling | `/docs/connect/connection-pooling` |
| Serverless Driver | `/docs/serverless/serverless-driver` |
| JavaScript SDK | `/docs/reference/javascript-sdk` |
| API Reference | `/docs/reference/api-reference` |
| TypeScript SDK | `/docs/reference/typescript-sdk` |
| Python SDK | `/docs/reference/python-sdk` |
## Framework and Language Guides
```bash
# Next.js
curl -H "Accept: text/markdown" https://neon.tech/docs/guides/nextjs
# Django
curl -H "Accept: text/markdown" https://neon.tech/docs/guides/django
# Drizzle ORM
curl -H "Accept: text/markdown" https://neon.tech/docs/guides/drizzle
# Prisma
curl -H "Accept: text/markdown" https://neon.tech/docs/guides/prisma
```
## Best Practices
1. **Always verify** - When answering questions about Neon features, APIs, or configurations, fetch the relevant documentation to verify your response is accurate.
2. **Check llms.txt first** - If you're unsure which documentation page covers a topic, fetch the llms.txt index to find the relevant URL. Don't make up URLs.
3. **Docs are the source of truth** - If there's any conflict between your training data and the documentation, the documentation is correct. Neon features and APIs evolve, so always defer to the current docs.
4. **Cite your sources** - When providing information from the docs, reference the documentation URL so users can read more if needed.
@@ -0,0 +1,57 @@
# What is Neon
Neon is a serverless Postgres platform designed to help you build reliable and scalable applications faster. It separates compute and storage to offer modern developer features such as autoscaling, branching, instant restore, and scale-to-zero.
For the full introduction, fetch the official docs:
```bash
curl -H "Accept: text/markdown" https://neon.tech/docs/introduction
```
## Core Concepts
Understanding Neon's resource hierarchy is essential for working with the platform effectively.
| Concept | Description | Key Relationship |
| ---------------- | --------------------------------------------------------------------- | ------------------------- |
| Organization | Highest-level container for billing, users, and projects | Contains Projects |
| Project | Primary container for all database resources for an application | Contains Branches |
| Branch | Lightweight, copy-on-write clone of database state | Contains Databases, Roles |
| Compute Endpoint | Running PostgreSQL instance (CPU/RAM for queries) | Attached to a Branch |
| Database | Logical container for data (tables, schemas, views) | Exists within a Branch |
| Role | PostgreSQL role for authentication and authorization | Belongs to a Branch |
| Operation | Async action by the control plane (creating branch, starting compute) | Associated with Project |
## Key Differentiators
1. **Serverless Architecture**: Compute scales automatically and can suspend when idle
2. **Branching**: Create instant database copies without duplicating storage
3. **Separation of Compute and Storage**: Pay for compute only when active
4. **Postgres Compatible**: Works with any Postgres driver, ORM, or tool
## Documentation Resources
| Topic | Documentation URL |
| ---------------------- | --------------------------------------------------------- |
| Introduction | https://neon.tech/docs/introduction |
| Architecture | https://neon.tech/docs/introduction/architecture-overview |
| Plans & Billing | https://neon.tech/docs/introduction/about-billing |
| Regions | https://neon.tech/docs/introduction/regions |
| Postgres Compatibility | https://neon.tech/docs/reference/compatibility |
```bash
# Fetch architecture docs
curl -H "Accept: text/markdown" https://neon.tech/docs/introduction/architecture-overview
# Fetch plans and billing
curl -H "Accept: text/markdown" https://neon.tech/docs/introduction/about-billing
```
## When to Use Neon
Neon is ideal for:
- **Serverless applications**: Functions that need database access without managing connections
- **Development workflows**: Branch databases like code for isolated testing
- **Variable workloads**: Auto-scale during traffic spikes, scale to zero when idle
- **Cost optimization**: Pay only for active compute time and storage used