This paper The Key to Effective UDF Optimization: Before Inlining, First Perform Outlining implements its ideas in DuckDB as an extension and I was curious specifically on how the authors access the SQL UDF to start working on it. I got the code from Sam Arch’s github , and I followed the instructions there on building and running their extension. In udf_transpiler_extension.cpp in the…
Abstract In part 1, I applied the prefix idea from Umbra-style strings to molecules and improved exact match queries via short-circuiting in a duckdb extension, duckdb_rdkit. I then tried to add more data to the prefix to also enable short-circuiting for substructure matches. This caused a degradation in exact match performance. Here in part 2, I improved the prefix by fitting more useful data…
Ideas from Umbra-style/German-style strings applied to molecules speed up exact match queries on molecules in duckdb_rdkit The current implementation of is_exact_match for finding molecules in duckdb_rdkit uses the standard molecule comparision algorithm found in the RDKit Postgres extension and the RDkit SQLite extension, chemicalite [ 1 ]. Here, I apply ideas from Umbra-style/German-style…
In An Empirical Evaluation of Columnar Storage Formats by X. Zeng, et al., VLDB 2023 [ 1 ], the authors studied the Parquet and ORC file formats in order to inform the design of the next generation of columnar storage formats. They began by analyzing the parameter distribution of data sets, and used the insights to construct a benchmark against which they could test the file formats in a variety…
I am using GNU 11.3.0. I had some issues with clang Code is in examples/embedded-c++/main.cpp . Will use this to enter the Query function Built duckdb with debug mode: GEN=ninja make debug in the root directory of the project Go to examples/embedded-c++ and in CMakeLists.txt use the debug duckdb build: -link_directories(../../build/release/src) +link_directories(../../build/debug/src) Then, in the…
tmpdb is a embedded key value store based off bitcask and go-caskdb written in NodeJS. The code can be found here . Why The main reason I worked on this is because I am interested in database internals and I wanted to work on a project related to that. I also thought this could be used in something like AWS Lambda as a cache – where it could be used like a hash map that stores the values on disk.…
I have been trying to build an extension ( duckdb_rdkit ) for duckdb that gives it the ability to do cheminformatic work via RDKit . I’ve been following and learning from the duckdb extension spatial , the sqlite3 extension chemicalite , as well as the RDKit Postgres extension as a pattern for building the extension. Many thanks to their work. Working with molecules in a computer Working with…