על אודות
Almost two decays of experience as a software engineer and research engineer in various…
פעילות
12K עוקבים
ניסיון חינוך
צפו Yoel בחוויה המלאה
ראה את כותרת, קְבִיעוּת ועוד.
או
רישיונות והסמכות
-
Yoel Zeldes
Marquis Who's Who
שהונפקו יפוג
תְעוּדַת זֶהוּת אישורים 308f4ac4f5124001a7a92f897a5baa896189e143f158478784e63a684610c491
פרסומים
-
Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
In this report, we introduce the Gemini 2.X model family: Gemini 2.5 Pro and Gemini 2.5 Flash, as well as our earlier Gemini 2.0 Flash and Flash-Lite models. Gemini 2.5 Pro is our most capable model yet, achieving SoTA performance on frontier coding and reasoning benchmarks. In addition to its incredible coding and reasoning skills, Gemini 2.5 Pro is a thinking model that excels at multimodal understanding and it is now able to process up to 3 hours of video content. Its unique combination of…
In this report, we introduce the Gemini 2.X model family: Gemini 2.5 Pro and Gemini 2.5 Flash, as well as our earlier Gemini 2.0 Flash and Flash-Lite models. Gemini 2.5 Pro is our most capable model yet, achieving SoTA performance on frontier coding and reasoning benchmarks. In addition to its incredible coding and reasoning skills, Gemini 2.5 Pro is a thinking model that excels at multimodal understanding and it is now able to process up to 3 hours of video content. Its unique combination of long context, multimodal and reasoning capabilities can be combined to unlock new agentic workflows. Gemini 2.5 Flash provides excellent reasoning abilities at a fraction of the compute and latency requirements and Gemini 2.0 Flash and Flash-Lite provide high performance at low latency and cost. Taken together, the Gemini 2.X model generation spans the full Pareto frontier of model capability vs cost, allowing users to explore the boundaries of what is possible with complex agentic problem solving.
-
ComMer: a Framework for Compressing and Merging User Data for Personalization
Large Language Models (LLMs) excel at a wide range of tasks, but adapting them to new data, particularly for personalized applications, poses significant challenges due to resource and computational constraints. Existing methods either rely on exposing fresh data to the model through the prompt, which is limited by context size and computationally expensive at inference time, or fine-tuning, which incurs substantial training and update costs. In this paper, we introduce ComMer - Compress and…
Large Language Models (LLMs) excel at a wide range of tasks, but adapting them to new data, particularly for personalized applications, poses significant challenges due to resource and computational constraints. Existing methods either rely on exposing fresh data to the model through the prompt, which is limited by context size and computationally expensive at inference time, or fine-tuning, which incurs substantial training and update costs. In this paper, we introduce ComMer - Compress and Merge - a novel framework that efficiently personalizes LLMs by compressing users' documents into compact representations, which are then merged and fed into a frozen LLM. We evaluate ComMer on two types of personalization tasks - personalized skill learning, using the tweet paraphrasing dataset and the personalized news headline generation dataset from the LaMP benchmark, and knowledge-intensive, using the PerLTQA dataset. Our experiments demonstrate that in constrained inference budget scenarios ComMer achieves superior quality in skill learning tasks, while highlighting limitations in knowledge-intensive settings due to the loss of detailed information. These results offer insights into trade-offs and potential optimizations in multi-document compression for personalization.
כות בים נוספים
-
Standing on the Shoulders of Giant Frozen Language Models
Huge pretrained language models (LMs) have demonstrated surprisingly good zero-shot capabilities on a wide variety of tasks. This gives rise to the appealing vision of a single, versatile model with a wide range of functionalities across disparate applications. However, current leading techniques for leveraging a "frozen" LM -- i.e., leaving its weights untouched -- still often underperform fine-tuning approaches which modify these weights in a task-dependent way. Those, in turn, suffer…
Huge pretrained language models (LMs) have demonstrated surprisingly good zero-shot capabilities on a wide variety of tasks. This gives rise to the appealing vision of a single, versatile model with a wide range of functionalities across disparate applications. However, current leading techniques for leveraging a "frozen" LM -- i.e., leaving its weights untouched -- still often underperform fine-tuning approaches which modify these weights in a task-dependent way. Those, in turn, suffer forgetfulness and compromise versatility, suggesting a tradeoff between performance and versatility. The main message of this paper is that current frozen-model techniques such as prompt tuning are only the tip of the iceberg, and more powerful methods for leveraging frozen LMs can do just as well as fine tuning in challenging domains without sacrificing the underlying model's versatility. To demonstrate this, we introduce three novel methods for leveraging frozen models: input-dependent prompt tuning, frozen readers, and recursive LMs, each of which vastly improves on current frozen-model approaches. Indeed, some of our methods even outperform fine-tuning approaches in domains currently dominated by the latter. The computational cost of each method is higher than that of existing frozen model methods, but still negligible relative to a single pass through a huge frozen LM. Each of these methods constitutes a meaningful contribution in its own right, but by presenting these contributions together we aim to convince the reader of a broader message that goes beyond the details of any given method: that frozen models have untapped potential and that fine-tuning is often unnecessary.
כות בים נוספים
-
Technical Report: Auxiliary Tuning and its Application to Conditional Text Generation
We introduce a simple and efficient method, called Auxiliary Tuning, for adapting a pre-trained Language Model to a novel task; we demonstrate this approach on the task of conditional text generation. Our approach supplements the original pre-trained model with an auxiliary model that shifts the output distribution according to the target task. The auxiliary model is trained by adding its logits to the pre-trained model logits and maximizing the likelihood of the target task output. Our method…
We introduce a simple and efficient method, called Auxiliary Tuning, for adapting a pre-trained Language Model to a novel task; we demonstrate this approach on the task of conditional text generation. Our approach supplements the original pre-trained model with an auxiliary model that shifts the output distribution according to the target task. The auxiliary model is trained by adding its logits to the pre-trained model logits and maximizing the likelihood of the target task output. Our method imposes no constraints on the auxiliary architecture. In particular, the auxiliary model can ingest additional input relevant to the target task, independently from the pre-trained model's input. Furthermore, mixing the models at the logits level provides a natural probabilistic interpretation of the method. Our method achieved similar results to training from scratch for several different tasks, while using significantly fewer resources for training; we share a specific example of text generation conditioned on keywords.
כות בים נוספים
-
Analysis of Product Presentation Impact on Voice Shopping Experience
AMLC
Explored the importance of optimizing product titles for Alexa Voice User Interface shopping by measuring the effectiveness of a product title based on its length and fluency.
-
Deep density networks and uncertainty in recommender systems
KDD (Knowledge Discovery and Data Mining) - best paper award (AdKDD & Target Ad workshop)
Building robust online content recommendation systems requires learning complex interactions between user preferences and content features. The field has evolved rapidly in recent years from traditional multi-arm bandit and collaborative filtering techniques, with new methods employing Deep Learning models to capture non-linearities. Despite progress, the dynamic nature of online recommendations still poses great challenges, such as finding the delicate balance between exploration and…
Building robust online content recommendation systems requires learning complex interactions between user preferences and content features. The field has evolved rapidly in recent years from traditional multi-arm bandit and collaborative filtering techniques, with new methods employing Deep Learning models to capture non-linearities. Despite progress, the dynamic nature of online recommendations still poses great challenges, such as finding the delicate balance between exploration and exploitation. In this paper we show how uncertainty estimations can be incorporated by employing them in an optimistic exploitation/exploration strategy for more efficient exploration of new recommendations. We provide a novel hybrid deep neural network model, Deep Density Networks (DDN), which integrates content-based deep learning models with a collaborative scheme that is able to robustly model and estimate uncertainty. Finally, we present online and offline results after incorporating DNN into a real world content recommendation system that serves billions of recommendations per day, and show the benefit of using DDN in practice.
כות בים נוספים
-
On-line fair allocations based on bottlenecks and global priorities
ICPE (International Conference on Performance Engineering)
System bottlenecks, namely those resources which are subjected to high contention, constrain system performance. Hence effective resource management should be done by focusing on the bottleneck resources and allocating them to the most deserving clients. It has been shown that for any combination of entitlements and requests a fair allocation of bottleneck resources can be found, using an off-line algorithm that is given full information in advance regarding the needs of each client. We extend…
System bottlenecks, namely those resources which are subjected to high contention, constrain system performance. Hence effective resource management should be done by focusing on the bottleneck resources and allocating them to the most deserving clients. It has been shown that for any combination of entitlements and requests a fair allocation of bottleneck resources can be found, using an off-line algorithm that is given full information in advance regarding the needs of each client. We extend this result to the on-line case with no prior information. To this end we introduce a simple greedy algorithm. In essence, when a scheduling decision needs to be made, this algorithm selects the client that has the largest minimal gap between its entitlement and its current allocation among all the bottleneck resources. Importantly, this algorithm takes a global view of the system, and assigns each client a single priority based on his usage of all the resources; this single priority is then used to make coordinated scheduling decisions on all the resources. Extensive simulations show that this algorithm achieves fair allocations according to the desired entitlements for a wide range of conditions, without using any prior information regarding resource requirements. It also follows shifting usage patterns, including situations where the bottlenecks change with time.
כות בים נוספים
פטנטים
-
Using compressed representations to adapt generative models to new context data
שהונפקו 20260093991
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing a task. In one aspect, a method comprises: receiving a query for a task to be performed; receiving a plurality of context content items for the task; for each content item of the plurality of content items, processing an input comprising a representation of the content item using a trained compression model to generate a compressed representation of the content item comprising one…
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing a task. In one aspect, a method comprises: receiving a query for a task to be performed; receiving a plurality of context content items for the task; for each content item of the plurality of content items, processing an input comprising a representation of the content item using a trained compression model to generate a compressed representation of the content item comprising one or more vectors of a fixed size; generating, using the compressed representations, an aggregated compressed representation comprising one or more vectors that represents the plurality of content items; and processing an input comprising (i) the query and (ii) the aggregated compressed representation using a generative neural network to generate a response to the query.
-
Reading assistant
שהונפקו WO2023215260A1
AI21's Wordtune Read patent. I was responsible for training the summarization models, the topic-guided summarization models, the topic extraction models, and the parsing of PDFs into structured text.
ממציאים נוספים
-
Modular reasoning, knowledge, and language systems
שהונפקו WO2023200762A1
AI21's Modular Reasoning, Knowledge and Language (MRKL) system patent. The patent introduces a groundbreaking architecture that decomposes a single query to a Large Language Model into a framework of specialized expert models. I contributed by designing and implementing novel architectures for some of the expert models, which were detailed in a paper we published - Standing on the Shoulders of Giant Frozen Language Models.
ממציאים נוספים
פרויקטים
-
AnotherDatum
-להציג
A technical blog about machine learning and data science.
Over 100k page views, featured in various publications.
Code can be found here: https://github.com/yoel-zeldes/yoel-zeldes.github.io.
כבוד ופרסים
-
Best paper award
-
Got a best paper award for the paper I led and published at KDD: Deep density networks and uncertainty in recommender systems.
In the paper we proposed a new neural network architecture that integrates uncertainty predictions for more efficient exploration of new recommendations in recommender systems. -
Magna Cum Laude
Hebrew University Of Jerusalem
In addition, included in Dean's List Award twice (in second and third years).