In the rapidly evolving world of AI, finding the right model for financial analysis can make or break your investment strategy. I put several top models to the test, comparing their performance and cost-effectiveness to see which one truly delivers the best value. This benchmark aims to evaluate various AI models’ ability to interpret and analyze financial news in the context of a specific…
Navigating the fast-paced world of global financial markets demands more than just keeping up—it requires insight. The sheer volume of information from financial news sources can be overwhelming. What if there was a way to not just sort through the noise but distill key insights. Try FinSummarizer ! FinSummarizer We introduce a fun little project FinSummarizer that blends technology with financial…
Today, we’re diving into the fascinating world of Mini-Dolly. We all know how incredible LLM models are, but their massive size can be a bit of a downer – requiring tons of resources and storage space. That’s where Mini-Dolly comes to the rescue! It’s a nifty experiment that’s all about getting the best of both worlds – top-notch performance without taking up your whole hard drive. Mini-Dolly…
I recently stumbled upon Gartner’s Hype Cycle Builder , which simply put allows you to build custom Gartner’s hype cycle graphs for a given technology innovation or architecture. Unfortunately, it’s only available for paying clients :dollar: Poor Man’s Gartner’s Hype Cycle Builder ! So I set out to build a poor man’s version of it and see if all technology trends follow this hype cycle. Before…
I had the pleasure of giving a talk in the first Nantes Scala Meetup to be hosted in the new Talend offices in Nantes. I’ve been playing with Scio for sometime when they asked me if I’m interested in giving a Scala talk, and since the previous talk I gave was about how Talend Pipeline Designer uses Apache Beam under the hood I decided to work on a talk that shows the pros of using a Scala Beam…
I’ve been a loyal follower of Data Eng Weekly newsletter (formerly Hadoop Weekly) for the past 6 years, the newsletter is a great source for everything related to Big data and data engineering in general with a wide selection of technical articles along with product announcements and industry news. For this year’s holidays side project I decided to analyze Data Eng’s archives , that go back to…
I had the pleasure to give a talk during the first european Apache Beam summit held in London. I presented Talend Pipeline Designer (previous Talend Data Streams) and how we built a modern Big Data pipeline designer with Apache Beam and finished the talk with a quick demo. The overall summit was pretty good and I had the chance to meet great people from the Apache Beam community. Video : Talend…
Crypto currencies Crypto currencies are gaining more and more momentum, and the last Bitcoin surge (17/12/2017) where the exchange price almost hit the 20000$ mark [1] is just another evidence of this momentum. Developers and users alike are riding the crypto currency wave with Bitcoin adoption soaring as evidenced by the number of unique address transactions [2]. But Bitcoin isn’t the only player…
We’ve been using OpenStack for a while and so recently we decided to contribute back some of our work to the OpenStack Sahara project. OpenStack Sahara aims to make it easy to do data processing on OpenStack, the project currently supports spawning Hadoop clusters and running data processing jobs on data in the OpenStack Object Store (OpenStack Switf). OpenStack Cinder Data Locality The first…
Processing network capture files is one of the several use cases of large scale processing in Hadoop using MapReduce. Network capture files record network activity by listening to interfaces and capturing network packets then storing the packet in a file. In a busy environment network edge nodes (routers, firewalls, Intrusion Detection Systems, …) generate gigabytes of network capture files per…
Apache Pig provides a simple to use abstraction layer on top of Hadoop MapReduce. Pig allows us to define complex data flows using its scripting language Pig-Latin. Pig-Latin is a procedural scripting language with a gradual learning curve and a lot of built-in functions and a fairly large open source community. A data flow, written in Pig-Latin, hides all of the low level complexities of…
Apache Hive provides a data warehousing layer on top of Hadoop. Hive uses Hadoop’s MapReduce as its query engine to execute complex SQL-like queries over data in Hadoop’s file system (HDFS). Hive was developed at Facebook to deal with the ever increasing volume of data in their data warehouses. It’s still in use at Facebook as the recently published blog post speaks of numerous innovations in…
Hadoop 2.4.0 was released last week and with it came the first part of the HDFS heterogeneous storage support. The idea behind HDFS heterogeneous storage is to expose multiple storage types (HDD, SDD, …) in HDFS and give HDFS clients the ability to choose a prefered storage type for their files. There’s an excellent post about it on Hortonworks blog . The complete implementation won’t be ready…