Adewumi Sunkanmi D.

University of California… • 6K followers

The original Bf-Tree paper repository is now open. Not my work, but I presented it at Berkeley systems reading group last December. (https://lnkd.in/gqDUnP6X). Typically, pages in the buffer pool are mapped to the size of the unit of disk I/O i.e 4KB, buffer pool page beyond 4KB causes torn writes, not uncommon in Postgres, MySQL, e.t.c. but handled meticulously. Caching a 4KB page waste memory if only one record is hot. On the other hand, caching records alone does not help for range query since there is no *spatial locality*, i.e no guarantee that records in the cache constitute all the records within that range in the leaf page. Caching the entire leaf page i.e 4KB gives you *spatial locality* i.e you know all the content (records) of the page and their true order. Some databases handles this by allowing users keep both record and block cache e.g RocksDB. However, this introduces data duplication, cache coherence complexities and the paper also mentioned static partitioning of memory for each cache which is difficult to tune correctly as one cache might be underutilized while the other is starved. Bf-Tree introduces mini pages (an in memory page mapped to individual leaf pages on disk) which is dynamically sized according to the current workload. Mini page size is initially sized as 64 bytes (cache line size i.e smallest unit of data that can be transferred between DRAM and the CPU cache) and then grows dynamically. Point reads are probabilistically cached (20% by default), if key is not found, phantom record is inserted in mini page to prevent wasteful reads in the future. Insertion are absorbed into the mini pages (a new mini page is created if none exist for the leaf page). Read-Modify check the mini page before checking leaf page on disk, if found, update stays in mini page to serve future reads faster. Tombstone is used for deletion. Mini page size is doubled (beginning at 64 bytes) and when it grows beyond 2KB, it is merged with the content of leaf page on disk to become a 4KB mini-page that mirrors the leaf page (good for range scan). For range scan, the mini page might not contain all the keys therefore it has to load the entire page from the disk, it doesn't immediately cache the entire page in memory to prevent bloating the mini page with a single unrepeated range scan. To mitigate disk I/O, Bf-Trees allows a frequently scanned mini-page to grow to full page size i.e 4KB preserving spatial locality. Some limitations I see (not in the paper): First, the assumption for optimizing range scan is that lots of range scans will visit the same subset of pages but if many range scan access pages stochastically, I/O gets hurt. Secondly, cache misses reads the entire page (1st I/O), the page is read again during merge (2nd I/O) and then write the page to disk during cache eviction (3rd I/O), this would have been two I/O in normal B-Tree. No free lunch :) #database #rust

CompuFlair LLC

360 followers

🚀 ML Project to Practice 📊 Student Performance Prediction Where to Practice (Run the Project): 📚Google Colab Notebook: https://lnkd.in/gzaViEh4 ❗ Problem: There are many reasons students might perform poorly in their studies, and perhaps we can prevent it. 🤔 🎯 Goal: Predict if a student might perform badly and help them achieve better scores. 📈 🛠️ Approach: Use Linear Regression and evaluate how good our model is using Loss Functions. 📊 -------------------------- 🚀 Master Machine Learning 🔬 A Physics-Based Approach 💼 Get Hired or Promoted 📈 👉 Here’s your roadmap: 1. 🔗 Visit the program page: https://lnkd.in/gWwq78Vp 2. 🎬 Start from Episode 1 and move through the episodes step by step 👣. 1. 🔹 Read the Research Notes and watch the embedded videos to deeply understand each concept. 2. 🔹 Take the Course. Open the corresponding Colab notebook, run the code from top to bottom, and see how these physics-based ideas map onto conventional ML algorithms. 3. 🔹 Run the Project. Apply your knowledge on real datasets, solve real-world problems, and gain hands-on experience that bridges theory and practice. 4. 🌟 Ready to go even further? ✨ Turn the project into a unique-to-you professional GitHub repository — with a clean, industry-standard folder structure, production-ready settings, and depoly as a web application to showcase your work live. 📩 Click Request Professional Setup on the page to get started. 🚀 FREE-or-Land-Job 6 Months BootCamp From Graduation in a Math-Based Field to a Data Science Career ✨ Program Highlights 💼 Land a Job or Pay Almost Nothing! 🧭 Step-by-Step Roadmap: 100% done-for-you structured schedule outlining exactly what to learn at each stage. 🧑‍🏫 Weekly Live Sessions: Expert-led project-solving sessions to master core data science skills through hands-on experience. 🌐 Build Your Online Presence & Portfolio: Work on real-world projects, receive expert reviews, and showcase your work on GitHub. 🧠 Technical & Soft Skills: Master all skills needed to land a job, from machine learning to interview and resume prep. 📄 Job Application Prep & Mentoring: Step-by-step guide on job search strategies, networking, and application materials. 🤝 Get Support When Needed: Connect with mentors for help, ask questions anytime, and join group discussions to learn and grow together. 🎯 Apply now: 🔗 https://lnkd.in/gDv64aFg

Alexandre Morgand, PhD

Simulon • 12K followers

When monocular depth estimation emerged, we quickly saw some refocusing application appearing but with diffusion models we can now really get real sharp information to enable many optical effects! National Yang Ming Chiao Tung University and University of Maryland, College Park present "Generative Refocusing: Flexible Defocus Control from a Single Image" Depth-of-field control is essential in photography, but getting the perfect focus often takes several tries or special equipment. Single-image refocusing is still difficult. It involves recovering sharp content and creating realistic bokeh. Current methods have significant drawbacks. They need all-in-focus inputs, depend on synthetic data from simulators, and have limited control over aperture. They introduce Generative Refocusing, a two-step process that uses DeblurNet to recover all-in-focus images from various inputs and BokehNet for creating controllable bokeh. Their main innovation is semi-supervised training. This method combines synthetic paired data with unpaired real bokeh images, using EXIF metadata to capture real optical characteristics beyond what simulators can provide. Their experiments show we achieve top performance in defocus deblurring, bokeh synthesis, and refocusing benchmarks. Additionally, our Generative Refocusing allows text-guided adjustments and custom aperture shapes. Also, Jia-Bin Huang (one of the author!) videos are gems. Don't hesitate to checkout his youtube channel (link below with everything!). #machinelearning #photography #monoculardepth #diffusionmodels #deeplearning #generativeAI #vfx #imagemanipulation