Harshvardhan Vatsa

I'm an ML researcher focused on representation learning, mechanistic interpretability, and understanding state space model architectures. Currently:

  • AI Intern at Rowboat Labs (YC S24)
  • AI Resident at Lossfunk, Bangalore

I got into ML through a friend who showed me CNNs, then drifted from fine-tuning LLMs into research. I'm drawn to trenching into the unknown — the parts where there isn't a tutorial yet.

My current research interests:

  • Representation learning in sequence models
  • Mechanistic interpretability of SSMs
  • State space model architectures (SSMs, Mamba)

Long-term I care about world models, embodied AI, and the science of why architectures work.

Outside research, I make music, take photographs, play an unreasonable amount of Minecraft, and build software in my spare time.

I also write blogs. Feel free to check them out here.

Say Hi on twitter, or email me at harshvardhanvatsa@gmail.com for collaborations or enquiries.

Email  /  Resume  /  LinkedIn  /  GitHub  /  twitter

profile photo

Machine Learning Research
Investigating Transformer-Based Architectures for Efficient Multi-Class Dental Disease Classification

Harshvardhan Vatsa, Hitesh Shivkumar, Bhavya Bhardwaj, Dr. Shalini L.

Trained mainly ViT and MobileViT model to 92.21% and 93.81% accuracy respectively on 11,653 dental images spanning 6 disease categories. Conclusion was that the MobileViT slightly outperforms traditional ViT on small dataset.

Wrote the code and setup the whole pipeline for preprocessing as well as training. Currently under review and gonna be published soon!

Open‑Source Contributions
Open Climate Fix

  • Analysed the TZ‑SAM solar dataset and fixed issues in the forecast predictor pipeline.
  • Analysed the TZ‑SAM solar dataset and fixed issues in the forecast predictor pipeline.
  • Modified Dockerfile and pyproject.toml to resolve Docker build failures.
Unsloth AI

  • Converted and published GGUF files for FLUX.1‑Kontext‑dev to improve accessibility.
  • Released files and docs to the repo; the Hugging Face release has 100k+ lifetime downloads.

Experience
Rowboat Labs (YC S24) | AI Intern — Jun 2026–Present

  • Working on AI research and engineering at an early-stage YC-backed startup.
Lossfunk | AI Resident — 2025–Present (Bangalore, India)

  • Researching representation learning and mechanistic interpretability of state space model architectures.
Mecha Systems | Machine Learning Intern — 2024 (Remote)

  • Cold-pitched the project that led to the internship offer.
  • Curated an 8,000-command Linux dataset and fine-tuned Llama-3.2-1B with QLoRA to run under 1 GB RAM, making it edge-deployable.
  • Built a Linux Command Assistant Agent for Mecha Comet, focused on Debian and Mechanix OS.

Projects
Steel Defect Detection
PyTorch, Computer Vision — Mar 2025

  • Developed a U‑Net architecture from scratch in PyTorch, featuring an encoder‑decoder with skip connections for precise pixel‑wise defect detection on steel surfaces.
  • Implemented a combined DICE‑BCE loss to address class imbalance and created a custom data pipeline using PIL and NumPy for efficient preprocessing of the Severstal dataset.
  • Attained strong performance metrics: 88.13% pixel accuracy, 0.7038 mean Dice coefficient, and 0.5841 mean IoU.
English–Swedish Neural Machine Translation
PyTorch, NLP — Jun 2025

  • Built a complete Transformer architecture from scratch with multi‑head attention and positional encoding.
  • Trained a custom tokenizer for English–Swedish translation and implemented a full training/validation pipeline.

This website was produced from a template made by Jon Barron.