Projects

Predict the trajectory of a self-driving vehicle, given a bird’s-eye view of the traffic scene. Used Lyft’s prediction dataset and the L5Kit Jupyter notebook templates, combining the attention mechanism of a Vision Transformer with a fully connected layer to predict the trajectory.

Parallelized obstacle collision checks with CUDA kernels.

1. Build the camera trajectory from a continuous image stream (KITTI Vision dataset).
2. Plot the trajectory and the triangulated 3D feature points in a map for visualization (Pangolin viewer).

1. Improved accuracy over the SOTA PoseFormer architecture by using various loss functions for human pose estimation.
2. DA-STS-GCN (Domain Adaptive Space-Time-Separable Graph Convolutional Network) for human pose forecasting.
3. Trained the networks on the Human3.6M dataset.

Projecting a 3D unit cube into a monocular video, using an ArUco marker as the feature location.

1. Find the lanes on the road.
2. Identify and mark the left and right lanes with different colors.

Path planning from a start to a goal location while avoiding obstacles.

Path planning from a start to a goal location while avoiding obstacles.

A robot arm planner implemented with ROS and MoveIt.

Fit a second-order polynomial to 2D data points of a ball following projectile motion.

1. Undistort two images with the given camera extrinsics.
2. Match image patches, keeping one image as the reference.
3. Create a disparity map using a sliding-window method.
4. Compute depth from the disparity map.

Image segmentation using k-nearest neighbours.

A 6-DOF arm can manipulate objects in SE(3) (translation + rotation), but one constraint remains: the arm’s reachability. Omnidirectional wheeled mobile robots can move in any direction in the XY plane, allowing more freedom than non-holonomic car-like robots. Combining the two gives a high-DOF arm with unlimited reach in the XY plane, useful for warehouse operations and agricultural tasks.

Technical approach

• Arm: inverse kinematics using the pseudo-inverse Jacobian.
• Base: wheel angular velocities by inverting the forward transformation from the base frame to the wheel frames, with PID feedback control on position error.

Inverse kinematics for a 6-DOF robotic arm, computed from the Jacobian pseudo-inverse of the forward transformation of each joint. Implemented in the CoppeliaSim simulator.

Multiple agents push a large object. Each agent plans a path to the destination using the RRT* algorithm.

Using a 128×128 RGB image to predict the free space on the ground for a wheeled mobile robot navigating its environment.

• Neural network: a small U-Net trained on a custom dataset, with an inference time of 7 ms.
• Classical vision (2 ms, shown in the video):
  1. Sample points near the robot and segment the image.
  2. Track the points across frames with optical flow and compute a homography.
  3. Filter out points that do not follow the homography of the points closest to the robot.
  4. Remove segments with a low density of points.

An Android phone’s accelerometer and gyroscope data are used to detect potholes in real time.

A simple shooting game written for an 11th-grade school project.