Current research

Topics I am studying and prototyping in 2026.

KV-Cache Optimization for Efficient LLM Inference2026

  • Studying Key-Value (KV) cache mechanisms in autoregressive transformer inference, and how caching prior attention states avoids redundant recomputation and reduces latency in LLMs.
  • Exploring memory-reduction techniques including Multi-Query Attention (MQA), Grouped-Query Attention (GQA) and quantized KV-cache representations to lower GPU memory use in long-context generation.
  • Analyzing trade-offs between cache eviction strategies (sliding window, attention-sink retention) and generation quality for extended context lengths.
Transformer AttentionMemory-Bound InferencePyTorch

GLM Architecture & Novel Model Design2026

  • Studying the General Language Model (GLM) architecture's autoregressive blank-infilling pretraining objective, which unifies language understanding and generation in one framework.
  • Exploring hybrid attention and encoder-decoder design variations to improve bidirectional context modeling while keeping autoregressive generation efficient.
  • Prototyping architectural modifications such as mixture-of-experts routing and modified positional encodings to improve scalability and long-context reasoning.
General Language Model (GLM)Architecture InnovationDeep Learning

Principal Component Analysis for Dimensionality Reduction2026

  • Applying Principal Component Analysis (PCA) to reduce the dimensionality of high-dimensional feature sets while preserving maximum variance for downstream model training.
  • Evaluating explained-variance thresholds and eigenvalue decomposition to balance information retention against model complexity and overfitting risk.
PCAFeature EngineeringHigh-Dimensional Data

Want to talk?

Reach out about roles, projects or research.

Get in touch