Mingfu Liang

• Models multi-stage cascade retrieval and re-ranking as a single tool-assisted reasoning LLM, rather than a pipeline of separately trained stages.
• Trained with reinforcement learning (DAPO) to optimise recommendation quality and token budget together — which doubles as joint feature selection across the whole cascade.
• First-author work on serving reasoning-based ranking at production scale.

• Targets re-ranking — the funnel stage closest to what the user actually sees — which most LLM-for-recommendation work skips in favour of retrieval and ranking.
• Combines mid-training on semantic IDs (a tokenizer with ≥99% uniqueness, so billions of non-semantic catalogue IDs enter the LLM vocabulary), reasoning traces distilled from a stronger teacher, and RL with verifiable rewards built for re-ranking.
• Delivers +18.7% R@1, +7.1% R@3, and +9.6% N@3 over legacy baselines on industrial-scale traffic.
• Two findings that shaped the system: supervised fine-tuning collapses at industrial scale (replaced by On-Policy Distillation), and LLMs will hack re-ranking rewards by preserving the incoming order or exploiting position bias — motivating conditional verifiable rewards.

• Replaces GR2's autoregressive decoder with a block-diffusion language model, decoding many reasoning positions in parallel over a few denoising steps instead of one forward pass per token.
• Closes the two accuracy gaps that a naive AR-to-diffusion conversion opens, via conversion fine-tuning, on-policy distillation, and reinforcement learning.
• Reaches a 2.4–3.5× throughput gain on the Amazon Beauty benchmark — the reasoning trace, not the ranking, was the bottleneck.

• A reasoning-augmented multimodal agent that unifies otherwise separate recommendation tasks behind one interface.

• A long-horizon driving scene generation framework. At its core is a novel RGB-D diffusion model that iteratively generates sparse, geometrically consistent keyframes, serving as reliable anchors for the scene's appearance and geometry.
• To maintain long-range geometric consistency, the model (1) jointly handles image and depth in a shared latent space, (2) explicitly conditions on the existing scene geometry (i.e., rendered point clouds) from previously generated keyframes, and (3) steers the sampling process with a warp-consistent guidance.
• AutoScape generates realistic and geometrically consistent driving videos of over 20 seconds, improving the long-horizon FID and FVD scores over the prior state of the art by 48.6% and 43.0%, respectively.

• The first work exploring Incremental Object Keypoint Learning: scaling up closed-set keypoint estimators with novel keypoints by incremental learning without forgetting, under either sufficient or limited label regimes, for medical and natural images.

• Created the first Automatically Improved Data Engine (AIDE) to scale up the Autonomous Vehicle (AV) system on handling safety-critical novel object detection.
• AIDE composes automatic issue identification, efficient data curation, model improvement via auto-labeling, and verification through diverse AV scenario generation, all intelligently powered by recent advancements in VLMs and LLMs.
• AIDE can automatically enable closed-set object detectors to detect novel objects, outperforming state-of-the-art (SOTA) open-vocabulary object detectors (OVOD) by a large margin, and achieves the highest cost-efficiency (training and/or labeling) among learning paradigms such as fully-supervised, semi-supervised, and active learning. It also maintains and even improves known-object detection performance, improving SOTA OVOD by >4% average precision (AP) on novel categories without human labels.
• Iterative use of AIDE significantly boosts performance, and minimal human feedback can lead to substantial gains — e.g., an extra 5.5% AP from correcting 30 images in the verification stage. AIDE also scales up with more unlabeled data and approaches the fully-supervised upper bound.

• Studied a novel and practical setting for incremental learning, i.e., incrementally learning to classify unseen subpopulations (e.g., different breeds of dogs) as their corresponding population (i.e., the class dog), without retaining the images used for learning the seen population — called Incremental Subpopulation Learning (ISL).
• Empirically showed that ISL is promising for alleviating the subpopulation shifting problem (i.e., the large performance drop, mostly >30%, when a model tests directly on unseen subpopulations), without sacrificing the original performance on the seen population.
• Proposed a two-stage learning framework as a novel and the first baseline tailored to ISL, which disentangles knowledge acquisition and forgetting to better handle the stability–plasticity trade-off, inspired by generalized Boosting Theory. Proposed novel proxy estimations to approximately measure forgetting and knowledge acquisition, creating a new optimization objective for ISL.
• Benchmarked the representative and state-of-the-art non-exemplar-based methods on a recently proposed large-scale dataset tailored to real-world subpopulation shifting for the first time, i.e., the BREEDS datasets, with extensive empirical study and formal analysis.

• We empirically show that the intrinsic stiffness phenomenon (SP) found in high-precision solutions of ordinary differential equations (ODEs) also widely exists in high-performance neural networks.
• We formally demonstrate that the Self-Attention Mechanism (SAM) is a stiffness-aware step size adaptor that can enhance a model's representational ability to measure intrinsic SP by refining the estimation of stiffness information and generating adaptive attention values, providing a new understanding of why and how SAM benefits model performance.
• This novel perspective can also explain the lottery ticket hypothesis in SAM, design new quantitative metrics of representational ability, and inspire a new theory-inspired approach, StepNet.

• Propose a task-oriented active (TOA) VQA method to actively and progressively acquire visual information according to the task, imitating the human cognitive process of reasoning, hypothesis, and verification.
• TOA consists of an LLM as the scheduler to recurrently make dynamic task planning and decisions like a human, and a visual executor to carry out the scheduler's orders. The LLM interacts with the vision modality progressively through multi-round dialogue to dynamically generate task-oriented plans based on previous interaction experience.
• The LLM analyses the question and the initial caption returned by the vision executor, and then either makes a hypothesis or requests more information based on its initial understanding and the open-world knowledge obtained during LLM pretraining.
• The LLM then dynamically adjusts its hypothesis based on previous conversations and the visual evidence obtained by the vision executor. This hypothesis–reasoning–verification procedure repeats, letting the LLM activate its internal open-world knowledge and proactively engage with the image to gather relevant information.

• An embodied agent that decides when it has seen enough to commit to a recognition, and when to keep moving and gather more evidence.
• Uses evidential uncertainty rather than softmax confidence, so the agent stays prudent on genuinely unfamiliar objects.
• Targets the open world, where the correct answer is sometimes “none of the known classes”.

• Borrows the attention mechanism from deep learning to improve numerical solvers when training data is scarce.

• Serves a trillion-parameter foundation model to online ads ranking without paying its full inference cost at serving time.
• Decouples the heavy external model from the latency-bound production stack, so the two can scale on independent schedules.

• Adds multi-head early exit to a RAG-enhanced recommender so easy queries stop early and hard ones keep computing.
• Trades accuracy against latency along a tunable curve rather than committing to one fixed operating point.

• Searches for how many vision tokens each layer actually needs instead of pruning them by a hand-set ratio.
• Cuts multimodal LLM inference cost while holding accuracy, with no retraining of the base model.

• Benchmarks associative reasoning in multimodal models while explicitly controlling for ambiguous prompts.
• Shows that ambiguity, not association ability, explains much of the apparent gap in prior evaluations.

• Distils a multimodal recommender through a variational information bottleneck, keeping only what transfers.
• Improves robustness to noisy and missing modalities at recommendation time.

• Steers continual segmentation training toward flat minima, which resist forgetting as new classes arrive.

• We show that performing classifier guidance on the latent space modelled by diffusion models can provide effective compositional generation, and also enables sequential editing with high fidelity.

Related lecture on “Guidance for Diffusion Models” given in CS-496 Deep Generative Models — slides.

• We propose a novel-yet-simple framework that shares an attention module throughout different network layers to encourage the integration of layer-wise information, called the Dense-and-Implicit-Attention (DIA) unit.
• Many choices of modules can be used in the DIA unit. Since Long Short Term Memory (LSTM) has the capacity to capture long-distance dependency, we focus on the case when the DIA unit is a modified LSTM (called DIA-LSTM).
• Experiments on benchmark datasets show that the DIA-LSTM unit is capable of emphasizing layer-wise feature interrelation and leads to significant improvement in image classification accuracy.
• We further empirically show that DIA-LSTM has a strong regularization ability for stabilizing the training of deep networks, demonstrated by experiments with the removal of skip connections or Batch Normalization in the whole residual network.

• We offer a new point of view that the self-attention mechanism can help regulate noise by enhancing instance-specific information to obtain a better regularization effect.
• We propose an attention-based BN called Instance Enhancement Batch Normalization (IEBN) that recalibrates the information of each channel by a simple linear transformation.
• IEBN has a good capacity for regulating batch noise and stabilizing network training to improve generalization, even in the presence of two kinds of noise attacks during training.

• Work done during a summer research internship at Nanyang Technological University (NTU, Singapore).
• The first work to explore contextual information for guiding channel pruning tailored to semantic segmentation.
• We formulate the embedded contextual information by leveraging layer-wise channel interdependency via the Context-aware Guiding Module (CAGM), and introduce Context-aware Guided Sparsification (CAGS) to adaptively identify informative channels in the cumbersome model by inducing channel-wise sparsity on the scaling factors in batch normalization layers.
• The resulting pruned models require significantly fewer operations for inference while maintaining comparable performance to (at times outperforming) the original models. We evaluated our framework on widely used benchmarks and showed its effectiveness on both large and lightweight models.

• Extends the lottery ticket hypothesis to self-attention modules inside residual CNNs, finding sparse attention subnetworks that train to full accuracy.

• Shares one attention module across layers instead of learning a separate one per layer, cutting parameters at equal accuracy.

• Combines several pruning criteria rather than trusting one, since no single criterion wins across layers.

• Alternates the training objective to settle in flat minima, which forget less as tasks arrive in sequence.

• Searches where to insert attention modules rather than attaching one to every block, keeping the gains at a fraction of the cost.