Skip to content

Commit cab6610

Browse files
committed
Add Alembic and constraint reflection coverage
1 parent 1f5ce0b commit cab6610

13 files changed

Lines changed: 604 additions & 5 deletions

‎README.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,25 @@ pytest -m integration
142142

143143
集成测试覆盖 SQLAlchemy Core、事务回滚、批量插入、ORM CRUD、元数据反射和连接池基础复用。
144144

145+
没有安装 pytest 的数据库主机也可以运行轻量探针:
146+
147+
```bash
148+
GAUSSDB_TEST_URL='gaussdb+gaussdb://user:password@host:port/postgres' \
149+
python scripts/run_integration_probe.py
150+
```
151+
152+
探针覆盖主键、唯一约束、普通索引反射、序列默认值和 Alembic Operations。
153+
154+
## Alembic 支持
155+
156+
测试依赖中包含 Alembic。本项目会在 Alembic 可用时注册 `gaussdb` DDL 实现,复用 Alembic PostgreSQL 基础实现,当前已验证:
157+
158+
- `Operations.create_table()`
159+
- `Operations.add_column()`
160+
- `Operations.drop_table()`
161+
162+
Alembic autogenerate 还需要更多真实场景回归。
163+
145164
## 打包
146165

147166
```bash

‎docs/GaussDB差异清单.md‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
# GaussDB 与 PostgreSQL 兼容差异清单
2+
3+
本文记录项目在真实 GaussDB 环境验证时发现的 SQLAlchemy 方言差异。Windows 实机验证和 GaussDB 505.1 专项验证需另行补充。
4+
5+
## 已发现并处理
6+
7+
### 1. 版本字符串格式
8+
9+
GaussDB 返回格式示例:
10+
11+
```text
12+
gaussdb (GaussDB Kernel 507.0.0 build ...)
13+
```
14+
15+
SQLAlchemy PostgreSQL 方言默认只识别 `PostgreSQL x.y.z` 或 `EnterpriseDB x.y.z`。
16+
17+
处理方式:
18+
19+
- 解析并保留真实内核版本到 `dialect.gaussdb_server_version_info`
20+
- `dialect.server_version_info` 使用保守 PostgreSQL 兼容版本,避免触发不兼容的新系统表查询
21+
22+
### 2. 默认 SQL_ASCII 导致文本返回 bytes
23+
24+
部分环境默认 `client_encoding=SQL_ASCII`,底层 `gaussdb` DB-API 会将文本列返回为 `bytes`。
25+
26+
处理方式:
27+
28+
- 方言默认传入 `client_encoding=UTF8`
29+
- 允许用户通过连接串显式覆盖
30+
31+
### 3. PostgreSQL 新版本系统表字段不兼容
32+
33+
直接复用 SQLAlchemy PostgreSQL 反射查询时,可能访问当前 GaussDB 不支持或不兼容的字段/表达式:
34+
35+
- `pg_attribute.attgenerated`
36+
- `pg_attribute.attidentity`
37+
- `pg_type.typcollation` 相关查询
38+
39+
处理方式:
40+
41+
- 覆盖 `get_columns()` 和 `get_multi_columns()`
42+
- 使用保守的 `pg_class`、`pg_namespace`、`pg_attribute`、`pg_attrdef` 查询完成基础列反射
43+
44+
### 4. 索引反射返回值与 PostgreSQL 方言预期不一致
45+
46+
SQLAlchemy PostgreSQL 方言在索引反射时会处理 PostgreSQL 特有的 index flag。真实 GaussDB 环境中该路径返回的 flag 类型与 SQLAlchemy 预期不一致,导致位运算失败。
47+
48+
处理方式:
49+
50+
- 覆盖 `get_indexes()`
51+
- 使用 `pg_index`、`pg_class`、`pg_attribute` 直接反射普通索引和唯一索引列
52+
53+
### 5. 约束反射采用保守查询
54+
55+
为了避免 PostgreSQL 新版本系统表字段差异,主键和唯一约束反射使用保守查询。
56+
57+
处理方式:
58+
59+
- 覆盖 `get_pk_constraint()`
60+
- 覆盖 `get_unique_constraints()`
61+
62+
真实环境还发现当前 GaussDB 不支持 PostgreSQL 风格的 `unnest(...) with ordinality`。约束和索引反射改用 `attnum = any(...)` 的保守写法。
63+
64+
### 6. HSTORE 不应默认启用
65+
66+
HSTORE 是 PostgreSQL 扩展,不应假设轻量化集中式环境可用。
67+
68+
处理方式:
69+
70+
- 默认关闭 `use_native_hstore`
71+
72+
### 7. Alembic 不认识 gaussdb 方言名
73+
74+
Alembic 的 DDL 实现按 SQLAlchemy `dialect.name` 查找。`gaussdb` 是第三方方言名,默认不在 Alembic 注册表中。
75+
76+
处理方式:
77+
78+
- 增加 `gaussdb_sqlalchemy.alembic`
79+
- 注册 `GaussDBImpl`
80+
- 继承 Alembic PostgreSQL DDL 实现以支持基础 Operations
81+
82+
## 已纳入集成测试
83+
84+
- SQLAlchemy Core 建表、插入、查询、删表
85+
- 事务回滚
86+
- 批量插入
87+
- ORM CRUD
88+
- 列元数据反射
89+
- 主键、唯一约束、普通索引反射
90+
- 序列和 `nextval()` 默认值
91+
- Alembic Operations 建表、加列、删表
92+
- 连接池基础复用
93+
94+
其中主键、唯一约束、普通索引、序列默认值和 Alembic Operations 也可通过 `scripts/run_integration_probe.py` 在没有 pytest 的数据库主机上验证。
95+
96+
## 待继续验证
97+
98+
- GaussDB 505.1 专项环境
99+
- Windows 实机 DLL/PATH/客户端加载
100+
- SSL 连接
101+
- 更多数据类型:`numeric`、`timestamp`、`date`、`boolean`、`text`、`bytea`
102+
- Alembic autogenerate 差异检测
103+
- 复杂索引、表达式索引、分区表、视图反射

‎docs/常见问题.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,10 +66,23 @@ pytest -m integration
6666
- 批量插入
6767
- ORM CRUD
6868
- 元数据反射
69+
- 主键、唯一约束、普通索引反射
70+
- 序列和 `nextval()` 默认值
71+
- Alembic Operations
6972
- 连接池基础复用
7073

7174
真实环境中,GaussDB 内核版本会保存在 `dialect.gaussdb_server_version_info`;`dialect.server_version_info` 使用保守的 PostgreSQL 兼容版本,避免 SQLAlchemy 误用 PostgreSQL 新版本系统表字段。
7275

76+
## 6. Alembic 能直接用吗?
77+
78+
当前已注册 `gaussdb` Alembic DDL 实现,基础 Operations 已通过真实库验证:
79+
80+
- 建表
81+
- 加列
82+
- 删表
83+
84+
复杂迁移和 autogenerate 仍建议在目标库上单独回归。
85+
7386
仍需继续验证:
7487

7588
- Windows 实机连接

‎docs/验证记录.md‎

Lines changed: 14 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -130,7 +130,18 @@ SQLAlchemy PostgreSQL 方言会根据 `server_version_info` 选择反射查询
130130

131131
- 增加 GaussDB 505.1 专项验证
132132
- 增加 Windows 真实环境验证
133-
- 增加 SQLAlchemy ORM 集成测试
134-
- 增加 metadata reflection 测试
133+
- 扩展 SQLAlchemy ORM 集成测试
134+
- 扩展 metadata reflection 测试
135135
- 增加 Alembic 迁移验证
136-
- 梳理 GaussDB 与 PostgreSQL 在系统表、数据类型、自增、序列、索引和约束上的差异
136+
- 持续梳理 GaussDB 与 PostgreSQL 在系统表、数据类型、自增、序列、索引和约束上的差异
137+
138+
## 后续建议处理状态
139+
140+
- GaussDB 505.1 专项验证:待手动验证
141+
- Windows 真实环境验证:待手动验证
142+
- SQLAlchemy ORM 集成测试:已增加基础 CRUD 覆盖
143+
- metadata reflection 测试:已增加列、主键、唯一约束、索引反射覆盖
144+
- Alembic 迁移验证:已增加 Operations 建表、加列、删表覆盖
145+
- 差异梳理:已新增 `docs/GaussDB差异清单.md`
146+
- 序列和默认值:已增加 `create sequence` 与 `nextval()` 验证
147+
- 轻量真实库探针:已新增 `scripts/run_integration_probe.py`

‎pyproject.toml‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,12 +32,13 @@ dependencies = [
3232

3333
[project.optional-dependencies]
3434
test = [
35+
"alembic>=1.13",
3536
"pytest>=8",
3637
]
3738

3839
[project.urls]
39-
Homepage = "https://github.com/example/gaussdb-sqlalchemy-driver"
40-
Issues = "https://github.com/example/gaussdb-sqlalchemy-driver/issues"
40+
Homepage = "https://github.com/jarrenL/GaussDB-Python-Driver"
41+
Issues = "https://github.com/jarrenL/GaussDB-Python-Driver/issues"
4142

4243
[project.entry-points."sqlalchemy.dialects"]
4344
gaussdb = "gaussdb_sqlalchemy.dialect:GaussDBDialect_gaussdb"

‎scripts/run_integration_probe.py‎

Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
"""Run a live GaussDB SQLAlchemy integration probe.
2+
3+
The script expects GAUSSDB_TEST_URL or --url and creates temporary tables with
4+
``codex_*`` prefixes. It is intentionally framework-free so it can run on a
5+
database host even when pytest is not installed.
6+
"""
7+
8+
from __future__ import annotations
9+
10+
import argparse
11+
import os
12+
import uuid
13+
14+
from sqlalchemy import Column
15+
from sqlalchemy import Index
16+
from sqlalchemy import Integer
17+
from sqlalchemy import MetaData
18+
from sqlalchemy import String
19+
from sqlalchemy import Table
20+
from sqlalchemy import UniqueConstraint
21+
from sqlalchemy import create_engine
22+
from sqlalchemy import inspect
23+
from sqlalchemy import text
24+
25+
26+
def table_name(prefix: str) -> str:
27+
return f"{prefix}_{uuid.uuid4().hex[:12]}"
28+
29+
30+
def assert_true(condition: bool, message: str) -> None:
31+
if not condition:
32+
raise AssertionError(message)
33+
34+
35+
def run(url: str) -> tuple[str, ...]:
36+
from alembic.migration import MigrationContext
37+
from alembic.operations import Operations
38+
39+
engine = create_engine(url, pool_pre_ping=True)
40+
results: tuple[str, ...] = ()
41+
42+
table = table_name("codex_idx_ut")
43+
index_name = f"ix_{table}_name"
44+
unique_name = f"uq_{table}_code"
45+
metadata = MetaData()
46+
Table(
47+
table,
48+
metadata,
49+
Column("id", Integer, primary_key=True),
50+
Column("code", String(32), nullable=False),
51+
Column("name", String(32), nullable=False),
52+
UniqueConstraint("code", name=unique_name),
53+
Index(index_name, "name"),
54+
)
55+
try:
56+
metadata.create_all(engine)
57+
inspector = inspect(engine)
58+
pk = inspector.get_pk_constraint(table)
59+
assert_true(pk["constrained_columns"] == ["id"], f"unexpected pk: {pk}")
60+
61+
uniques = inspector.get_unique_constraints(table)
62+
assert_true(
63+
any(
64+
constraint["name"] == unique_name
65+
and constraint["column_names"] == ["code"]
66+
for constraint in uniques
67+
),
68+
f"unexpected unique constraints: {uniques}",
69+
)
70+
71+
indexes = inspector.get_indexes(table)
72+
assert_true(
73+
any(
74+
index["name"] == index_name and index["column_names"] == ["name"]
75+
for index in indexes
76+
),
77+
f"unexpected indexes: {indexes}",
78+
)
79+
finally:
80+
metadata.drop_all(engine)
81+
results += ("pk_unique_index",)
82+
83+
table = table_name("codex_seq_ut")
84+
sequence = f"{table}_id_seq"
85+
with engine.begin() as conn:
86+
conn.execute(text(f"drop table if exists {table}"))
87+
conn.execute(text(f"drop sequence if exists {sequence}"))
88+
conn.execute(text(f"create sequence {sequence} start 1"))
89+
conn.execute(
90+
text(
91+
f"create table {table} ("
92+
f"id int primary key default nextval('{sequence}'), "
93+
"name varchar(32))"
94+
)
95+
)
96+
conn.execute(text(f"insert into {table} (name) values (:name)"), {"name": "a"})
97+
conn.execute(text(f"insert into {table} (name) values (:name)"), {"name": "b"})
98+
rows = conn.execute(text(f"select id, name from {table} order by id")).all()
99+
assert_true(rows == [(1, "a"), (2, "b")], f"unexpected rows: {rows}")
100+
101+
columns = {column["name"]: column for column in inspect(conn).get_columns(table)}
102+
assert_true(
103+
"nextval" in columns["id"]["default"],
104+
f"unexpected default: {columns['id']}",
105+
)
106+
conn.execute(text(f"drop table {table}"))
107+
conn.execute(text(f"drop sequence {sequence}"))
108+
results += ("sequence",)
109+
110+
table = table_name("codex_alembic_ut")
111+
with engine.begin() as conn:
112+
context = MigrationContext.configure(conn)
113+
operations = Operations(context)
114+
operations.create_table(
115+
table,
116+
Column("id", Integer, primary_key=True),
117+
Column("name", String(32), nullable=False),
118+
)
119+
operations.add_column(table, Column("remark", String(64)))
120+
conn.execute(
121+
text(
122+
f"insert into {table} (id, name, remark) "
123+
"values (:id, :name, :remark)"
124+
),
125+
{"id": 1, "name": "created", "remark": "via alembic"},
126+
)
127+
row = conn.execute(
128+
text(f"select id, name, remark from {table} where id=:id"),
129+
{"id": 1},
130+
).one()
131+
assert_true(
132+
row == (1, "created", "via alembic"),
133+
f"unexpected alembic row: {row}",
134+
)
135+
operations.drop_table(table)
136+
results += ("alembic",)
137+
138+
return results
139+
140+
141+
def main() -> int:
142+
parser = argparse.ArgumentParser()
143+
parser.add_argument("--url", default=os.environ.get("GAUSSDB_TEST_URL"))
144+
args = parser.parse_args()
145+
if not args.url:
146+
parser.error("--url or GAUSSDB_TEST_URL is required")
147+
148+
results = run(args.url)
149+
print("integration probe ok:", ",".join(results))
150+
return 0
151+
152+
153+
if __name__ == "__main__":
154+
raise SystemExit(main())

‎src/gaussdb_sqlalchemy/__init__.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,9 @@
11
"""SQLAlchemy support for Huawei GaussDB."""
22

3+
from .alembic import register_alembic_impl
34
from .dialect import GaussDBDialect_gaussdb
45

6+
register_alembic_impl()
7+
58
__all__ = ["GaussDBDialect_gaussdb"]
69
__version__ = "0.1.0"

‎src/gaussdb_sqlalchemy/alembic.py‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
"""Alembic DDL integration for the GaussDB SQLAlchemy dialect."""
2+
3+
from __future__ import annotations
4+
5+
6+
def register_alembic_impl() -> bool:
7+
"""Register GaussDB with Alembic when Alembic is installed."""
8+
9+
try:
10+
from alembic.ddl.postgresql import PostgresqlImpl
11+
except Exception:
12+
return False
13+
14+
class GaussDBImpl(PostgresqlImpl):
15+
__dialect__ = "gaussdb"
16+
17+
return True

0 commit comments

Comments
 (0)