libduckdb.so name is different for compilation and runtime

[AmebaBrain]

With a new 1.1.3 release it's required to rename libduckdb.so library to libduckdb.1.1.3.so to have compilation succeeded.

# Rename library file with version number
mv "libduckdb.${LIB_EXT}" "libduckdb.${DUCKDB_VERSION}.${LIB_EXT}"

But in the same time, during runtime the compiled duckdb_fdw.so still expects the duckdb library to be named as libduckdb.so and not libduckdb.1.1.3.so. Otherwise, it fails with

psql:/docker-entrypoint-initdb.d/fdw_setup.sql:1: ERROR:  could not load library "/usr/lib/postgresql/17/lib/duckdb_fdw.so": libduckdb.so: cannot open shared object file: No such file or directory

Is it expected behavior? Use libduckdb.1.1.3.so for the compilation but libduckdb.so during runtime?

[alitrack]

compiled in this way

export DUCKDB_VERSION=1.1.3
make USE_PGXS=1 
make USE_PGXS=1 install 

[AmebaBrain]

but for this to work you also have to have physically file libduckdb.1.3.3.so. otherwise you will get compilation error. and once compilation is done, you will have to have original libduckdb.so file back again. otherwise, compiled duckdb_fdw.so won't work

[JosefMachytka-credativ]

Hello guys, thank you for your work on duckdb_db.
But I must confirm, @AmebaBrain is right, I also had to have both libduckdb.so and libduckdb.1.1.3.so files.
Without libduckdb.1.1.3.so file compilation did not work for me.
I would guess you have different libduckdb.x.x.x.so files on your testing environment so you can test different versions of duckdb.
And switch between them using env var for compilations.
Maybe you could look at it please?

[theory]

Note that the project Dockerfile does not work due to this issue:

$ docker build .
 > [builder 4/4] RUN cd duckdb_fdw    && export DUCK_ARCH=$(uname -m | sed -e s/arm64/aarch64/ | sed -e s/x86_64/amd64/)    && wget -c https://github.com/duckdb/duckdb/releases/download/v1.1.3/libduckdb-linux-${DUCK_ARCH}.zip    && unzip -o -d . libduckdb-linux-${DUCK_ARCH}.zip    && cp libduckdb.so $(pg_config --libdir)    && make USE_PGXS=1    && make install USE_PGXS=1:                                                                                                                                                                                          
0.083 --2025-01-16 21:44:51--  https://github.com/duckdb/duckdb/releases/download/v1.1.3/libduckdb-linux-aarch64.zip
0.088 Resolving github.com (github.com)... 140.82.112.4
0.110 Connecting to github.com (github.com)|140.82.112.4|:443... connected.
0.194 HTTP request sent, awaiting response... 302 Found
0.265 Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/138754790/2865358d-7169-4422-b53e-323490f7d704?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20250116%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20250116T214451Z&X-Amz-Expires=300&X-Amz-Signature=faee7f19b61b7ccb8a55bbb6cdc5b5162349396a02d3a29f5182d77e82fb7c83&X-Amz-SignedHeaders=host&response-content-disposition=attachment%3B%20filename%3Dlibduckdb-linux-aarch64.zip&response-content-type=application%2Foctet-stream [following]
0.265 --2025-01-16 21:44:51--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/138754790/2865358d-7169-4422-b53e-323490f7d704?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20250116%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20250116T214451Z&X-Amz-Expires=300&X-Amz-Signature=faee7f19b61b7ccb8a55bbb6cdc5b5162349396a02d3a29f5182d77e82fb7c83&X-Amz-SignedHeaders=host&response-content-disposition=attachment%3B%20filename%3Dlibduckdb-linux-aarch64.zip&response-content-type=application%2Foctet-stream
0.265 Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.111.133, 185.199.109.133, 185.199.110.133, ...
0.269 Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.111.133|:443... connected.
0.427 HTTP request sent, awaiting response... 200 OK
0.503 Length: 31056394 (30M) [application/octet-stream]
0.509 Saving to: ‘libduckdb-linux-aarch64.zip’
# wget output ellided
1.778 
1.779 2025-01-16 21:44:53 (23.3 MB/s) - ‘libduckdb-linux-aarch64.zip’ saved [31056394/31056394]
1.779 
1.783 Archive:  libduckdb-linux-aarch64.zip
1.783   inflating: ./libduckdb.so          
2.072   inflating: ./libduckdb_static.a    
2.370   inflating: ./duckdb.hpp            
2.376   inflating: ./duckdb.h              
2.405 gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wshadow=compatible-local -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -moutline-atomics -g -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -fvisibility=hidden -I. -I./ -I/usr/include/postgresql/17/server -I/usr/include/postgresql/internal  -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -I/usr/include/libxml2   -c -o connection.o connection.c
2.543 gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wshadow=compatible-local -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -moutline-atomics -g -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -fvisibility=hidden -I. -I./ -I/usr/include/postgresql/17/server -I/usr/include/postgresql/internal  -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -I/usr/include/libxml2   -c -o option.o option.c
2.626 gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wshadow=compatible-local -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -moutline-atomics -g -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -fvisibility=hidden -I. -I./ -I/usr/include/postgresql/17/server -I/usr/include/postgresql/internal  -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -I/usr/include/libxml2   -c -o deparse.o deparse.c
2.982 gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wshadow=compatible-local -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -moutline-atomics -g -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -fvisibility=hidden -I. -I./ -I/usr/include/postgresql/17/server -I/usr/include/postgresql/internal  -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -I/usr/include/libxml2   -c -o sqlite_query.o sqlite_query.c
3.070 gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wshadow=compatible-local -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -moutline-atomics -g -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -fvisibility=hidden -I. -I./ -I/usr/include/postgresql/17/server -I/usr/include/postgresql/internal  -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -I/usr/include/libxml2   -c -o duckdb_fdw.o duckdb_fdw.c
3.517 g++ -Wall -Wpointer-arith -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wshadow=compatible-local -Wformat-security -fno-strict-aliasing -fwrapv -moutline-atomics -g -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -std=c++11 -fPIC -fvisibility=hidden -fvisibility-inlines-hidden -I. -I./ -I/usr/include/postgresql/17/server -I/usr/include/postgresql/internal  -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -I/usr/include/libxml2   -c -o sqlite3_api_wrapper.o sqlite3_api_wrapper.cpp
6.007 gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wshadow=compatible-local -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -moutline-atomics -g -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC -fvisibility=hidden -shared -o duckdb_fdw.so connection.o option.o deparse.o sqlite_query.o duckdb_fdw.o sqlite3_api_wrapper.o -L/usr/lib/aarch64-linux-gnu  -Wl,-z,relro -Wl,-z,now -L/usr/lib/llvm-16/lib  -Wl,--as-needed -fvisibility=hidden -L. -lduckdb.1.1.3 -lstdc++
6.011 /usr/bin/ld: cannot find -lduckdb.1.1.3: No such file or directory
6.016 collect2: error: ld returned 1 exit status
6.017 make: *** [/usr/lib/postgresql/17/lib/pgxs/src/makefiles/../../src/Makefile.shlib:261: duckdb_fdw.so] Error 1
------

 1 warning found (use docker --debug to expand):
 - SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ENV "POSTGRES_HOST_AUTH_METHOD") (line 32)
Dockerfile:23
--------------------
  22 |     # build fdw
  23 | >>> RUN cd duckdb_fdw \
  24 | >>>    && export DUCK_ARCH=$(uname -m | sed -e s/arm64/aarch64/ | sed -e s/x86_64/amd64/) \
  25 | >>>    && wget -c https://github.com/duckdb/duckdb/releases/download/v${DUCKDB_VERSION}/libduckdb-linux-${DUCK_ARCH}.zip \
  26 | >>>    && unzip -o -d . libduckdb-linux-${DUCK_ARCH}.zip \
  27 | >>>    && cp libduckdb.so $(pg_config --libdir) \
  28 | >>>    && make USE_PGXS=1 \
  29 | >>>    && make install USE_PGXS=1
  30 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c cd duckdb_fdw    && export DUCK_ARCH=$(uname -m | sed -e s/arm64/aarch64/ | sed -e s/x86_64/amd64/)    && wget -c https://github.com/duckdb/duckdb/releases/download/v${DUCKDB_VERSION}/libduckdb-linux-${DUCK_ARCH}.zip    && unzip -o -d . libduckdb-linux-${DUCK_ARCH}.zip    && cp libduckdb.so $(pg_config --libdir)    && make USE_PGXS=1    && make install USE_PGXS=1" did not complete successfully: exit code: 2

[XBsleepy]

I build 1.1.2 from source, get ducked.so without version number.
I try to delete DUCKDB_VERSION in the Makefile.
When I try to CREATE EXTENSION ducked_fdw,errors occur:

 undefined symbol: _ZN6duckdb9ErrorDataC1ERKSs

[theory]

My solution to this was to create a symlink with the version in the name. Not ideal; would be great if duckdb_fdw just depended on libduckdb.so.