Follow The Rules: Online Signal Temporal Logic Tree Search for Guided Imitation Learning in Stochastic Domains

Robotics Institute Summer Scholar Program

Seamlessly integrating rules in Learning-from-Demonstrations (LfD) policies is a critical requirement to enable the real-world deployment of AI agents. Recently, Signal Temporal Logic (STL) has been shown to be an effective language for encoding rules as spatio-temporal constraints. This work uses Monte Carlo Tree Search (MCTS) as a means of integrating STL specification into a vanilla LfD policy to improve constraint satisfaction. We propose augmenting the MCTS heuristic with STL robustness values to bias the tree search towards branches with higher constraint satisfaction. While the domain-independent method can be applied to integrate STL rules online into any pre-trained LfD algorithm, we choose goal-conditioned Generative Adversarial Imitation Learning as the offline LfD policy. We apply the proposed method to the domain of planning trajectories for General Aviation aircraft around a non-towered airfield. Results using the simulator trained on real-world data showcase 60% improved performance over baseline LfD methods that do not use STL heuristics. I developed a motion planning algorithm using methods of behavior cloning. I used the current state, past trajectory, and intent to determine the subsequent actions with a discriminator reward based method. Using real-world aircraft flight data from the TrajAir dataset, augmented with the agent’s goal to generate optimal trajectories as future actions. Our system incorporates continuous learning allowing autonomous aircraft to integrate into regular manned traffic flow safely.

The key contributions in my work include:

1. Usage of Monte Carlo Tree Search that couples motion prediction with planning. MCTS rolls out our policy and selects the best trajectory plan based on variable metrics.
2. Utilization of Signal Temporal Logic as the formal logic mechanism to ensure constraints are modeled accurately and satisfied.
3. Implementing multi-agent MCTS search method for motion planning of multiple dynamic agents.

Figure shows the proposed approach in a prototypical scenario to plan for an aircraft landing in a non-towered airfield. The expert/pilot demonstrations (grey) are used offline to train an LfD policy; online, we use the signal temporal logic specifications in the MCTS expansion to ensure rule compliance.

Overview of the approach: offline, we train an LfD policy using datasets, which are used online in a monte-carlo tree search (MCTS). The online expansion uses a modified heuristic that uses robustness values from signal temporal logic (STL) specification to guide the search toward higher rule conformance.

• Duration: 2021-2022

Bounded distance control for Multi-UAV formation safety and preservation in target-tracking applications

Research Internship at the Guidance, Control, and Decision Systems Laboratory, Indian Institute of Science, Bangalore

The notion of safety in multi-agent systems assumes great significance in many emerging collaborative multi-robot applications. In this paper, we present a multi-UAV collaborative target-tracking application by defining bounded inter-UAV distances in the formation in order to ensure safe operation. In doing so, we address the problem of prioritizing specific objectives over others in a multi-objective control framework. We propose a barrier Lyapunov function-based distributed control law to enforce the bounds on the distances and assess its Lyapunov stability using a kinematic model. The theoretical analysis is supported by numerical results, which account for measurement noise and moving targets. Straight-line and circular motion of the target are considered, and results for quadratic Lyapunov function-based control, often used in multi-agent multi-objective problems are also presented. A comparison of the two control approaches elucidates the advantages of our proposed safe control in bounding the inter-agent distances in a formation.
A concluding evaluation using Robot Operating System simulations illustrates the practical applicability of the proposed control to a pair of multi-rotors visually estimating and maintaining their mutual separation within specified bounds as they track a moving target.

A two-agent formation with desired inter-agent distance, and associated lower and upper bounds is shown here

• Date: 2020-2021