Biography
Hi! I am Karmesh, a PhD student at Georgia Tech, advised by Prof. Dhruv Batra and Prof. Zsolt Kira. I am currently a research intern at Mistral AI, working on multimodal reasoning.
My research focuses on building agents that can reason, remember, and act in complex environments, with an emphasis on multimodal reasoning and embodied AI. Previously, I was an AI Resident at FAIR, working with the Habitat and Cortex team under the supervision of Dr. Oleksandr Maksymets and Prof. Batra. Before that, I worked as a Senior Robotics Engineer at ISEE, an autonomous vehicles startup working on automating yard trucks. I completed my Masters in Robotics Systems Development (MRSD) at the CMU Robotics Institute in 2020.
I am looking for full-time roles starting this fall. Please reach out if you think there may be a good fit.
Download my resumé.
- Multimodal Reasoning
- Agents
- Embodied AI
Ph.D. in Computer Science, 2026
Georgia Institute of Technology
Masters in Robotic Systems Development, 2020
Carnegie Mellon University
B.Tech in Mechanical Engineering, 2017
Indian Institute of Technology, Guwahati
Experience
- Initiated and led the Multimodal Reasoning effort, establishing end-to-end training, evaluation, and data pipelines.
- Developing rubric-guided reinforcement learning methods to reduce visual hallucinations and improve grounded reasoning in multimodal language models.
- Developed deployment pipelines for web agents to operate on real websites while effectively avoiding bot detection.
- Created data filtering pipeline and trained VLM-based agents for web navigation tasks.
- Researched self-supervised pretraining techniques for learning useful visual representations for embodied agents.
- Released the HM3D-Semantics (HM3DSem) dataset and the Open-Vocabulary Mobile Manipulation (OVMM) benchmark based on the Habitat Simulator.
- Explored deep uncertainty estimation techniques for predicting the closed loop tracking performance of an autonomous vehicle controller. Estimated the collision probability of the AV with respect to obstacles in an occupancy grid.
- Improved the trajectory optimization planner and robustified its collision checking. This led to an increased confidence in its performance and resulted in its deployment on the AV.
- Built toolboxes to automate the system identification and calibration procedure of ISEE’s vehicles.
- Researched and implemented various vehicle and tire models for control applications in autonomous vehicles.
- Worked on improving the localization module of an autonomous vehicle by fusing ORB-SLAM output with GPS, IMU, and wheel odometry.
