Add Amazon Redshift connector (#2) - #3
Conversation
- New connector src/connectors/redshift.py following the standard connect/list_tables/fetch_table pattern. Redshift is wire-compatible with PostgreSQL, so it reuses psycopg2 — no new dependencies - Amazon Redshift option in the Database Connectors UI (default port 5439) - Tests covering list_tables, fetch_table and the row limit using a stubbed connection, so they run without a live cluster - README: usage snippet, connector status table and roadmap updated
| from src.connectors.redshift import fetch_table | ||
| fetch_table("host", 5439, "dev", "awsuser", "password", "transactions", limit=500) | ||
| assert "LIMIT 500" in fake_connection.cursor().executed_query | ||
|
|
There was a problem hiding this comment.
def test_fetch_table_validates_table_name(self, fake_connection): from src.connectors.redshift import fetch_table with pytest.raises(ValueError, match="not found"): fetch_table("host", 5439, "dev", "awsuser", "password", "nonexistent_table")
There was a problem hiding this comment.
Added this test in 087c44c — it passes along with the rest of the suite.
harshitboots
left a comment
There was a problem hiding this comment.
Can do this which will give more liberty to user while connecting to any tables
| cursor.close() | ||
| conn.close() | ||
| return tables | ||
|
|
There was a problem hiding this comment.
` think we can use following code as its Doesn't work if user wants to query views or temporary tables (though they'd need to be in the whitelist query) as well now
actually it will resolve postgres and mysql issue also.
`def fetch_table(host: str, port: int, database: str, user: str, password: str, table: str, limit: int = 1000) -> pd.DataFrame:
"""Fetch a table from Redshift with SQL injection protection."""
conn = connect(host, port, database, user, password)
cursor = conn.cursor()
try:
# Validate table exists to prevent SQL injection
valid_tables = list_tables(host, port, database, user, password)
if table not in valid_tables:
raise ValueError(f"Table '{table}' not found in database. Available tables: {', '.join(valid_tables)}")
cursor.execute(f"SELECT * FROM {table} LIMIT {limit}")
columns = [desc[0] for desc in cursor.description]
rows = cursor.fetchall()
return pd.DataFrame(rows, columns=columns)
finally:
cursor.close()
conn.close()`
There was a problem hiding this comment.
Applied in 087c44c. fetch_table now validates the table name against list_tables() before querying and raises a ValueError listing the available tables if it's not found. Cursor/connection cleanup is now handled in a finally block. The same pattern would also apply to the Postgres and MySQL connectors — happy to address those in a follow-up PR.
Feature: Added DuckDB Connector (Resolves Issue #3)
- New connector src/connectors/duckdb_conn.py following the standard connect/list_tables/fetch_table pattern - DuckDB option in the Database Connectors UI (file path + table) - Tests covering list_tables, fetch_table and the row limit - README: usage snippet, connector status table and roadmap updated
Add DuckDB connector (#3)
What this adds
Amazon Redshift connector, as requested in the README's next-targets list (issue #2).
src/connectors/redshift.py— follows the standardconnect/list_tables/fetch_tablepattern used by the other connectors. Redshift is wire-compatible with PostgreSQL, so it connects viapsycopg2— zero new dependencies.Tests
Added
TestRedshiftConnectorwith three tests (list_tables,fetch_table, row limit). They monkeypatchpsycopg2.connectwith a stubbed connection, so the suite runs anywhere — no live cluster or AWS credentials required.All existing tests still pass.
README
Note
This branch is independent of #2's DuckDB work (both branch off main). If both are merged, the only overlap is the connector dropdown list in
app.py— a trivial one-line conflict.