With the introduction of ChatGPT and LLMs to the general masses, there has been an increased focus in vector search, which includes topics such as semantic search. In addition to specialized vector databases ( Vespa , Pinecone ) built specifically for such tasks, many existing data stores such as MongoDB , Lucene , and even PostgreSQL now support vector search. Vector Search Search engines such as…
When I first started the NoSQL NYC meetup back in 2010, it was the first NoSQL group of its kind on Meetup.com. The reasoning for creating the group was not because I was an expert on the subject, it was because I knew nothing. How can I learn more? So many new database types were being released around this time. Hadoop started small in 2006, CouchDB became an Apache subproject in 2008 and MongoDB…
There has been some discussion on the Elasticsearch mailing list lately about applying index-time boosts at the document level, aka document boosting. The practice has been discouraged (in fact, the Elasticsearch team has officially deprecated document boosting) in favor of query-time scoring, but without any detailed explanation why. Instead of repeating myself each time the question is asked, I…
If you have been in the workforce for a considerable amount of time, you might have a job profile on a job site or two. Although using one's network is perhaps one of the best ways to secure a new position, I found myself once again using a job site to find a new position in an area where I had no connections. For this job hunt, I posted my resume on Monster.com and Dice.com, two of the most…
A quick introduction on how to create a plugin in elasticsearch that allows you to define new REST endpoints. While working on a common issue that I have been repeatedly experiencing, I started thinking about if there could be a better way to tackle it. Good code should be flexible enough so that it is adaptable to many different situations and not hard-coded for a specific scenario (unless you a…
In my previous post , I implemented the algorithm for linear regression using gradient descent in Scala using two different methods: standard builtin mathematical methods and Scalala , a Scala linear algebra library. Shortly after writing the solution I started to wondering if using Scalala had any performance impact on the runtime cost of the solution. While Scalala does have the overhead of…
Implementing linear regression using gradient descent in Scala based on Andrew Ng's machine learning course. Stanford Univeristy recently announced that they will be offering thirteen online courses as part of their Stanford Engineering Everywhere program. According to the NYTimes , 58K people signed up for the Articial Intelligence course alone. Three days later, the number of interested students…
A while back, Meetup.com issued an pseudo-challenge: steal their database . Nothing that would result in the FBI knocking on your door mind you, but a look into their streaming API. Meetup.com streams all their public events and RSVPs via HTTP streaming or HTML5 websockets, so all the is required to steal their database is a connection to a stream and the ability to save the content. Their simple…
The NoSQL NYC Meetup has been enjoying quite a year. Great talks from all spectrums of the NoSQL world: document stores, graph databases, key-value stores, you name it. What I have learned from all these talks is that nothing wows a crowd more than a live demo. Marko Rodriguez 's demonstation of Gremlin using OpenRDF Sail data not even on his computer was particularly fascinating. That is why I am…