top of page


Learning to Swarm

Natural Swarm behaviors have fascinated researchers for decades. Early works focused on manually designing algorithms, which mimic realistic cohesive flocking behavior based on local information. Recent developments employ learning techniques, especially imitation learning requiring direct control input as supervision, to automatically train local controllers. However, we lack direct control supervision in general settings. In this work, we try to learn the decentralized controller based on pure observational swarm trajectory. Our framework learned the controller on 10-agent simulation and scale up to 50-agent swarm. 

3D Boids

2D Boids

3D Boids Scaling

Self-propelled Particles

Robust Metrics under Scaling swarm size


Video Instance Segmentation

Instance segmentation is a fundamental task in robotic perception. The technique is particularly important in recent autonomous vehicle perception development. Recent trends move from instance segmentation on images to videos. Feature points tracking identifies the flow of detected points over frames. Combining the instance segmentation on successive static frames and feature point detection provides a physical-aware video instance segmentation solution, which does not require a huge amount of extra memory compared to the SOTA video instance segmentation.

Gaussian Processes Regression on 1-D Temporal Data

Gaussian Processes are a line of Bayesian probabilistic models, which are popular in regression with uncertainty problems. The requirement of manually selected kernel function and extensive computation prevent the framework scale to universal and large-scale problems. In this work, we aim to propose a universal kernel design to handle versatile time-series data named grid spectral mixture kernel. We rely on a set of kernels sampled from kernel hyper-parameter space and learning the weights to combine them. Using an ADMM solver, we achieve the sparse combination of kernels, which in turn provides an interpretation of the dataset.



Pursuit-Evasion on Boats (Dec. 2021)

Formation Control (Dec. 2021)



bottom of page