Sanjay OrugantiResearch Scientist · Rensselaer Polytechnic Institute
Research
How can a robot be taught, and trusted, the way a new colleague is?
Today, changing what a robot does takes a specialist. I want the person who knows the job to be able to teach it.

OntoAgentic AI is the core system running HARMONIC. Try it here.
Teach OntoAgent on this page No LLMs inside
A pocket-size OntoAgent lives here. Tell it what you want. If it does not understand, it asks, and you can teach it. What you teach it stays in your browser.
This is the pocket edition. The real OntoAgent was developed by Sergei Nirenburg, Marjorie McShane, Jesse English, Sanjay Oruganti and many students at the LEIA Lab. Sanjay added the physical AI dimension to it: robots, through HARMONIC. Meet the real OntoAgent →
Things to try
Easter egg: type and watch.
What I work on
-
Cognitive robots as teammates
Robots that reason about goals and plans, talk with the people they work with, and act on real hardware.
LEIA Lab, RPI · 2023 – present
-
Learning from instruction
A person teaches a new procedure in plain language. The robot learns it, asks when it is unsure, and carries it out safely.
LEIA Lab, RPI · 2023 – present
-
Trust, explanation, and metacognition
Robots that check what they know before they act, and can explain what they did and why.
LEIA Lab, RPI · 2023 – present
-
Knowledge sharing in robot teams
Robots that pass skills to one another during a mission, by asking a teammate or by overhearing one.
HeRo Lab and DICE Lab, University of Georgia · 2017 – 2023
-
Biomedical and optical sensing devices
Portable devices that read scattered light to characterize aerosol particles and respiratory bacteria, with machine learning to interpret the readings.
DICE Lab, University of Georgia · 2019 – 2025
-
Self-organization and swarms
Earlier work on how order emerges in biological systems and in robot swarms.
University of Georgia · 2015 – 2022
Projects
-
HARMONIC
A cognitive architecture for robots that work in teams with people.
-
Why Cognitive Robotics Matters
Can a language model stand in for a cognitive architecture when a robot team’s task is safety-critical?
-
Metacognition and transparency
Robots that know the limits of what they know and show their reasoning to teammates.
-
Teaching through instruction
A person teaches the robot a new procedure in plain language.
-
Shipboard maintenance
A robot and a technician work out why an engine is overheating and replace the part.
-
Multi-robot search
A ground robot and a drone search for a lost object with a human operator.
-
IKT-BT
Robots that learn by overhearing their teammates.
-
KT-BT
Robots that teach each other new behaviors during a mission.
-
HeRoSwarm
A miniature open-source swarm robot, built with students I mentored.
-
Optical scatter device
A hand-sized device that reads scattered light to characterize particles and bacteria in breath.
Dissertation
Exploiting Emergence, Heterogeneity, and Knowledge Transfer in Multi-Agent Systems
Three studies of teams of unlike agents: how collective behavior emerges, how differences in what agents know change a mission, and how agents can pass knowledge to one another.