# Stanford Robotics Seminar ENGR319 | Autumn 2026 | Skydio: Autonomy for a Family of Flying Robots

## Executive summary

Skydio presented its comprehensive autonomy stack for flying robots, demonstrating capabilities ranging from remote, multi-drone orchestration to complex, autonomous missions in dense urban and critical infrastructure environments. The system integrates hardware (docking stations, drones like the F10 Lightrunner and X10 Plus) with advanced software that handles navigation, real-time object tracking, and mission planning. Key technical breakthroughs include using deep learning for proactive line-of-sight maintenance (Shadowing), fusing multiple sensor sources for reliable localization (Global Pose Estimate), and enabling open-vocabulary detection for specialized inspections and security monitoring.

## Key takeaways

- Autonomous Fleet Management: A single operator can remotely manage multiple drones across different geographical locations (e.g., headquarters and Colorado testing sites) from a single web interface. The system includes 'airspace deconfliction' to prevent collisions automatically, even when multiple drones are operating simultaneously.
- Advanced AI for Tracking and Visibility: The 'Shadowing' feature uses a deep learning model trained in Reinforcement Learning (RL) to predict visibility maps. This allows the drone to proactively maneuver to maintain line of sight to a subject (e.g., a car or person), even when the subject moves behind obstacles like buildings.
- Infrastructure Inspection and Mapping: For utility inspection, the system uses RTK attachments for highly reliable pose estimation. It performs 3D reconstruction and dense point cloud generation to identify assets and automatically generate optimal shot sheets for comprehensive inspection coverage.
- Scalability and Operational Resilience: The architecture is designed for 24/7 operation, transitioning from a 'toy' concept to critical infrastructure. The system utilizes 'connect fusion' by combining multiple communication sources (e.g., point-to-point links, 5G) to ensure low latency and high-resolution video streaming, even when connectivity is intermittent.

## Technical details

- Localization and Mapping: The system relies on combining multiple map sources (Digital Surface Map, Digital Terrain Map) and uses dense bundle adjustment and reconstruction (via top and bottom nav cameras) to update maps in real-time, overcoming the fundamental problem of outdated map data in construction zones.
- Global Pose Estimation (GPS Reliability): To counter unreliable GPS signals in urban canyons (due to multipath effects), the system fuses all sensor measurements (camera data, IMU, etc.) to build a Global Pose Estimate, significantly improving accuracy in challenging environments.
- Autonomy Stack and Hardware: The entire stack is built in-house, from the docking hardware to the low-level vehicle software and cloud software, allowing for optimization. Key hardware includes the F10 Lightrunner (fixed-wing) and the X10 Plus (modular, small form factor). The system utilizes a 'Megadock' containing a robotic arm for autonomous charging and launching.
- AI and Model Deployment: High-level reasoning tasks (like open-vocabulary detection) run in the cloud, while high-frequency, safety-critical tasks (like obstacle avoidance and subject tracking) run on the drone's onboard computer (e.g., NVIDIA Orin NX). The system is designed to handle complex transitions (e.g., vertical takeoff to forward flight) using MPC controllers.

## Practical implications

- Public Safety: Rapid deployment for 911 calls, providing critical information (e.g., identifying non-lethal situations) and improving response times (e.g., 52% of the time, the drone is the first thing to arrive).
- Utility/Critical Infrastructure: Proactive inspection of assets (e.g., power lines, substations) to identify damage early, preventing major outages.
- Security: Autonomous, persistent monitoring of large areas, capable of detecting specific objects (e.g., wooden pallets) or suspicious behavior.
- Operational Efficiency: Reducing reliance on human pilots by enabling high-level, autonomous mission planning and fleet orchestration.

## Topics

Autonomous Robotics, Computer Vision, Deep Learning, Critical Infrastructure, Drone Technology, Edge Computing, Stanford Robotics Seminar ENGR319, Skydio R1/R2, F10 Lightrunner, Skydio X10 Plus

Source: https://www.youtube.com/watch?v=Y3Y8pSOYOBo