Digital Robot Judge: Building a Task-centric Performance Database of Real-World Manipulation With Electronic Task Boards

Robotics aims to develop manipulation skills approaching human performance. However, skill complexity is often over- or underestimated based on individual experience, and the real-world performance gap is difficult or expensive to measure through in-person competitions. To bridge this gap, we propose a compact, Internet-connected, electronic task board to measure manipulation performance remotely; we call it the digital robot judge, or “DR.J.” By detecting key events on the board through performance circuitry, DR.J provides an alternative to transporting equipment to in-person competitions and serves as a portable test and data-generation system that captures and grades performances, making comparisons less expensive. Data collected are automatically published on a web dashboard (WD) that provides a living performance benchmark that can visualize improvements in real-world manipulation skills of robot platforms over time across the globe.