Autonomous Multi-Check AI Code Review
A Python tool that runs Claude Code in an autonomous, multi-check review loop with four levels of review thoroughness, cross-check coherence validation, and convergence detection.
Alexander Marquardt
Recent content on Alexander Marquardt
Latest posts
From messy product feeds to demo-ready e-commerce data
Turn messy open-source product data into clean, image-rich NDJSON for e-commerce demos and relevance work.
Ranking by Profit and Popularity in Elasticsearch
Moved to Elastic’s blog - https://www.elastic.co/search-labs/blog/function-score-query-boosting-profit-popularity-elasticsearch
Personalizing e-commerce search results based on purchase history in Elasticsearch
Introduction Whether you’re looking for a product in an online store, an article in a news archive, or a file in a company knowledge base, the quality of the search experience determines how quickly you find what you need. Behind the scenes, many of these systems are powered by Elasticsearch , a popular open-source search engine designed to handle large volumes of data and return relevant results…
Efficient bitwise matching of documents in Elasticsearch
This article has now been published on Elastic’s website. Please check it out at: https://www.elastic.co/search-labs/blog/efficient-bitwise-matching-in-elasticsearch
Automating the Import and Export of Kibana Saved Objects
Introduction Kibana is an open-source data visualization and exploration tool used for log and time-series analytics, application monitoring, and operational intelligence use cases. It offers powerful and easy-to-use features that allow users to visualize data from Elasticsearch in various formats such as charts, tables, and maps. While Kibana offers a robust user interface for managing many…
Barcelona: A Personal Guide
Motivation Originally from Canada, I have lived in Barcelona on and off since 2003. Over the years, many friends and acquaintances have asked for my recommendations on what to see and do in this beautiful city. To save time and ensure no detail is missed, I’ve compiled this comprehensive guide based on my personal experiences and research I’ve done over the years. Welcome to Barcelona!…
Re-directing Elasticsearch documents with out-of-range timestamps that (would) fail to get written into Time Series Data Streams
Introduction Elasticsearch Time Series Data Streams (TSDS) are designed to provide an efficient and scalable way to handle time-based data within the Elasticsearch ecosystem. This feature is specifically optimized for storing, searching, and managing time-series data such as metrics, and events, where data is continuously indexed in chronological order. However, if events arrive with timestamps…
Cut and paste a table from Google Sheets into an HTML document – no code required
Introduction In a recent article that I wrote for Airbyte it was necessary for me to copy and paste a table from Google Sheets into Webflow . Unfortunately, Webflow doesn’t currently support copy/paste of tables directly in its editor. In this blog, I present a workaround that can be used to convert a table from Google Sheets into HTML, which can then be written into a Webflow custom code block .…
Using Logstash to scan inside event contents to replace sensitive data with a consistent hash
Introduction Logstash is commonly used for transforming data before it is sent to another system for storage, and so it is often well positioned for finding and replacing sensitive text, as may be required for GDPR compliance. Therefore, in this blog I show how Logstash can make use of a ruby filter to scan through the contents of an event and to replace each occurrence of sensitive text with the…
A summary of recent research on increasing lifespan
Authors Ronald R. Marquardt Suzhen Li Alexander Marquardt In association with All Natural Nutritional Products (ANNP) Inc. Introduction This blog provides an overview of recent research in the field of ageing. A brief overview of the book: “Lifespan: Why we age — and why we don’t have to” David A. Sinclair and M. D. LaPlante have written a New York Times Bestseller called…
Combining Elasticsearch stemmers and synonyms to improve search relevance
This is now published on Elastic’s official blog. Please check it out at: https://www.elastic.co/blog/improve-search-relevance-by-combining-elasticsearch-stemmers-and-synonyms
Driving Filebeat data into separate indices (uses legacy index templates)
Introduction When driving data into Elasticsearch from Filebeat , the default behaviour is for all data to be sent into the same destination index regardless of the source of the data. This may not always be desirable since data from different sources may have different access requirements , different retention policies, or different ingest processing requirements. In this post, we’ll use…
Using Kibana's Painless Lab (Beta) to test an ingest processor script
Introduction In several previous blog posts I have shown how a Painless script can be used to process new documents as they are ingested into an Elasticsearch cluster. In each of these posts I have made use of the simulate pipeline API to test the Painless scripts. While developing such scripts, it may be helpful to use Painless Lab (Beta) in Kibana to debug Painless scripts. In this blog I will…
Using Elasticsearch Painless scripting to recursively iterate through JSON fields
Authors Alexander Marquardt Honza Kral Introduction Painless is a simple, secure scripting language designed specifically for use with Elasticsearch. It is the default scripting language for Elasticsearch and can safely be used for inline and stored scripts. In one of its many use cases, Painless can modify documents as they are ingested into your Elasticsearch cluster. In this use case, you may…
Understanding and fixing "too many script compilations" errors in Elasticsearch
Introduction When using Elasticsearch, in some rare instances you may see an error such as “Too many dynamic script compilations within X minutes”. Such an error may be caused by a poor script design where parameters are hard-coded . In other cases this may be due to the script cache being too small or the compilation limit being too low. In this article, I will show how to determine…
Using Logstash and Elasticsearch to calculate transaction duration in a microservices architecture
Introduction Elasticsearch allows you to unify your observability data in a powerful datastore so you can search and apply interactive analytics in real time to a huge number of use cases. In one such use case, you may be using Elasticsearch to monitor a system that is composed of multiple microservices that process a given transaction. For such a system, you may be collecting an event…
Using Grok with Elasticsearch to add structure to your data
This article is available on Elastic’s blog as a 3-part series. Please check it out at the following URLs: https://www.elastic.co/blog/structuring-elasticsearch-data-with-grok-on-ingest-for-faster-analytics https://www.elastic.co/blog/slow-and-steady-how-to-build-custom-grok-patterns-incrementally…
Storing ingest time and calculating ingest lag in Elasticsearch
This article is available on Elastic’s blog at: https://www.elastic.co/blog/calculating-ingest-lag-and-storing-ingest-time-in-elasticsearch-to-improve-observability
Using boolean queries to improve search relevancy in Elasticsearch
Page moved to https://alexmarquardt.com/using-boolean-queries-to-improve-search-relevancy-in-elasticsearch/
Using boolean queries to improve search relevance in Elasticsearch
This article is available at: https://www.elastic.co/blog/how-to-improve-elasticsearch-search-relevance-with-boolean-queries
Using slow logs in Elastic Cloud Enterprise
Authors Tom Schreiber Alex Marquardt Version This blog article is based on ECE 2.4.3. Introduction Elastic Cloud Enterprise (ECE) is a platform designed to ease the management, deployment, and configuration of multiple Elasticsearch clusters through a single administrative user interface. ECE, is the same product that powers the Elasticsearch Service hosted offering, and is available for…
Using Elastic machine learning to detect anomalies in derivative values
Introduction In this blog, we use Elastic machine learning (ML) and derivative aggregations to detect sudden unexpected increases or decreases in the rate-of-change of CPU load on servers that are monitored by Metricbeat . In order to make this blog easier to follow and the results easy to recreate, we abstract away the requirement for driving data from Metricbeat, and instead generate…
Using the Elasticsearch Enrich Processor with CSV data
This article is available at: https://www.elastic.co/blog/how-to-enrich-logs-and-metrics-using-an-elasticsearch-ingest-node
Converting CSV to JSON in Filebeat
Introduction Many organisations use excel files for creating and storing important data. For various reasons it may be useful to import such data into Elasticsearch. For example, one may need to get Master Data that is created in a spreadsheet into Elasticsearch where it could be used for enriching Elasticsearch documents . Or one may wish to use Elasticsearch and Kibana for analysing a dataset…
Enriching data with the Logstash translate filter
Introduction Logstash is an open source, server-side data processing pipeline that ingests data from a multitude of sources, transforms it, and then sends it to one or more outputs. One use of Logstash is for enriching data before sending it to Elasticsearch. Logstash supports several different lookup plugin filters that can be used for enriching data. Many of these rely on components that are…
How to create maintainable and reusable logstash pipelines
This article is available at: https://www.elastic.co/blog/how-to-create-maintainable-and-reusable-logstash-pipelines
Using Logstash and Elasticsearch scripted upserts to transform eCommerce purchasing data
Introduction Logstash is a tool that can be used to collect, process, and forward events to Elasticsearch. In order to demonstrate the power of Logstash when used in conjunction with Elasticsearch’s scripted upserts , I will show you how to create a near-real-time entity-centric index . Once data is transformed into an entity-centric index, many kinds of analysis become possible with simple…
Emulating transactional functionality in Elasticsearch with two-phase commits
Introduction Elasticsearch supports atomic create, update, and delete operations at the individual document level, but does not have built-in support for multi-document transactions . Although Elasticsearch does not position itself as a system of record for storing data, in some cases it may be necessary to modify multiple documents as a single cohesive unit. Therefore, in this blog post we…
Converting local time to ISO 8601 time in Elasticsearch
This article is available at: https://www.elastic.co/blog/converting-local-time-to-iso-8601-time-in-elasticsearch
Es Local Indexer - using Elasticsearch for searching locally stored documents
Moved to: https://alexmarquardt.com/es-local-indexer-desktop-search-built-with-elasticsearch/
ES Local Indexer – Desktop search powered by Elasticsearch
Introduction Elasticsearch provides search functionality for some of the most important websites in the world including Wikimedia (i.e. Wikipedia) , eBay , Yelp , Tinder , and many others. Elasticsearch is super scalable, which means that just as easily as it can be scaled it up for use in huge complex systems, it can also be scaled down for use in smaller projects. ES Local Indexer is a small…
How to mitigate hangovers
Introduction Many natural products exist that are purported to be able to prevent hangovers, but unfortunately when one talks about natural hangover remedies it is difficult to find many well funded scientific studies that validate or refute such claims. This is at least in part because few if any companies are willing to invest millions of dollars investigating and running clinical trials to…
Financial implications of exercising share options
Disclaimer I am not an accountant and this article should not be considered as financial or tax advice . I am providing analysis and calculations which may be used at your own peril. This article is written to demonstrate basic concepts, and does not account for country-specific tax laws or company-specific share option details. Your individual situation may invalidate some or all of the arguments…
Counting unique beats agents sending data into Elasticsearch
Introduction When using Beats with Elasticsearch, it may be useful to keep track of how many unique agents are sending data into an Elasticsearch cluster, and how many documents each agent is submitting. Such information for example could be useful for detecting if beats agents are behaving as expected. In this blog post, I first discuss how to efficiently specify a filter for documents…
Improving the performance of Logstash persistent queues
This article is available at: https://www.elastic.co/blog/using-parallel-logstash-pipelines-to-improve-persistent-queue-performance
Debugging Elasticsearch and Lucene with IntelliJ IDEA
This article can be found at: https://www.elastic.co/blog/how-to-debug-elasticsearch-source-code-in-intellij-idea
A step-by-step guide to enabling security, TLS/SSL, and PKI authentication in Elasticsearch
This article is available at: https://www.elastic.co/blog/elasticsearch-security-configure-tls-ssl-pki-authentication
How to tune Elasticsearch for aggregation performance
Introduction By default, Elasticsearch is tuned for the best trade-off between write performance and query performance for the majority of use cases. In this blog posting we cover some parameters that can be configured to improve query-time aggregation performance, with some of these improvements coming at the expense of write performance. Note that this blog posting does not present anything that…
Using Logstash to drive filtered data from a single source into multiple output destinations
This article is available at: https://www.elastic.co/blog/using-logstash-to-split-data-and-send-it-to-multiple-outputs
Using Logstash prune capabilities to whitelist sub-documents
Overview Logstash’s prune filter plugin can make use of whitelists to ensure that only specific desired fields are output from Logstash, and that all other fields are dropped. In this blog post we demonstrate the use of Logstash to whitelist desired fields and desired sub-documents before indexing into Elasticsearch. Example input file As an input to Logstash, we use a CSV file that contains…
Deduplicating documents in Elasticsearch
This article is available at: https://www.elastic.co/blog/how-to-find-and-remove-duplicate-documents-in-elasticsearch
Trade-offs to consider when storing binary data in MongoDB
Introduction When using MongoDB there are several approaches that make it easy to store and retrieve binary data, but it is not always clear which approach is the most appropriate for a given application. Therefore, in this blog post I discuss several methods for storing binary data when using MongoDB and the trade-offs associated with each method. Many of the trade-offs discussed here would…
How to generate unique identifiers for use with MongoDB
Now on MongoDB’s website This blog article has been published by MongoDB at https://www.mongodb.com/blog/post/generating-globally-unique-identifiers-for-use-with-mongodb Motivation By default, MongoDB generates a unique ObjectID identifier that is assigned to the _id field in a new document before writing that document to the database. In many cases the default unique identifiers assigned by…
How to manually perform a point in time restore in MongoDB
Introduction While MongoDB provides high-availability and data durability through automatic replication of data to multiple servers, this replication does not protect the database against human or application errors. For example, if an administrator drops a database, the drop operation will be replicated across the MongoDB deployment, and data will be deleted. If such an event occurs through human…
About the Author
Senior Principal Architect | GenAI & Search In my current role at Elastic, I work on GenAI and Search across real-world deployments where relevance, reliability, and performance requirements drive architectural decisions. I capture high-impact, one-off solutions and turn them into reusable assets, including patterns, middleware, tools, and delivery models. These assets reduce recurring escalations…
E-commerce Demo Assets
Public image assets intended for demos and testing.
Professional Record and Publications
My professional work spans semiconductor architecture, distributed search systems, and large-scale data platforms. This page provides supporting material for that work, including selected publications, patents, open-source projects, and formal education. External Profiles LinkedIn Profile Github Profile Recognition Author of three papers that were selected to be amongst the 25 most significant…