Xusheng LuoIRTA LAB · NC STATE
Experiments and simulations in planning, learning, and collaboration.
Coordinating who does what, when, and how to move—across robot arms, mobile platforms, and shared workspaces.
Four robot arms transfer objects across a shared workspace. Task ordering and collision-free motions are planned together, so each arm can prepare for the next handover.
Transfer the green object first, then the orange object, and finally the blue object. Coordinate handovers among the four arms while avoiding collisions in the cluttered workspace.
A chain of four arms transfers objects in a specified order, coordinating successive handovers.
Transfer the objects from the left side to the right side in order: yellow → blue → red. Use successive handovers along the four-arm chain.
Robot arms plan collaborative transfers around a wall while satisfying the task specification.
Move the yellow object to its target, then move the blue object, and finally bring the yellow object back, and avoid the red region. Note: The red region only needs to be avoided in the final move.
A team of arms selects among alternative delivery targets encoded in a temporal logic task.
Move the yellow object first. Then move either the blue or the red object to its corresponding target. The planner chooses which alternative to execute.
Four arms choose between two valid task specifications. The selected plan combines an obstacle-avoiding handover with a direct placement.
The planner selects Option 2: hand over the yellow object to reach Target 5, then place the blue object directly at Target 1.
The planner assigns an object to an arm that can reach the target directly. Compare this with the handover example.
Move the object from Target 1 to Target 2, satisfying Target 1 before Target 2. Both locations are reachable by one arm, so the task can be completed directly.
An object passes between two arms to reach its target. The planner reasons about both task assignment and reachability.
Move the object from Target 1 to Target 2, satisfying Target 1 before Target 2. The locations lie on opposite sides of the workspace, requiring a handover between arms.
A robot arm and a moving quadruped coordinate an object transfer, bringing mobile platforms into the planning problem.
Transfer the object from Target 1 to Target 2. Coordinate the arm and mobile robot to complete the ordered transfer. Note: The quadruped does not need to come to a complete stop.
Robot arms and a moving platform coordinate their motions to transfer objects through the workspace.
Complete three transfers in sequence: Target 1 → 4 (green), then Target 2 → 5 (red), then Target 3 → 6 (blue). Coordinate the two arms with the moving platform.
Turning human instructions into structured task specifications that robots can reason about and execute.
Natural-language instructions become hierarchical temporal logic specifications for multi-step manipulation.
Place the carrot on the pink plate, the orange on the blue plate, and the red apple on the yellow plate. Complete all three placements in any order.
A robot executes a multi-step household instruction with ordering requirements and alternative ways to complete the task.
Place the statues on the table; afterward, move a plate carrying a keychain to the table; then, turn on a floor lamp and pick up a watch. Also put a box containing a credit card on the couch, at any point relative to that sequence.
Hierarchical temporal logic connects task allocation with planning, organizing complex objectives for a robot team.
First, turn off the floor lamp. Then, move the box to a sofa and the laptop to an armchair. Finally, put the credit card in the garbage can. Allocate these actions across the robot team to minimize completion time.
Learning structured skills from demonstrations, steering robot behavior, and examining the reliability of learned perception.
Emergent Neural Automaton Policies learn symbolic task structure from visuomotor trajectories and connect it to low-level control.
Project : Learning the structure behind a skillReStruct reconfigures learned task structure at inference time to steer robot behavior while accounting for physical constraints.
Project : Steering behavior through task structureA preference steers the robot to push the T-shaped object toward the blue target first.
Project : Push-T: blue firstA preference steers the robot to push the T-shaped object toward the green target first.
Project : Push-T: green firstA preference steers the robot to orient the peg and insert its red end into the hole.
A preference steers the robot to orient the peg and insert its white end into the hole.
Simulation-aided learning from demonstration supports robotic LEGO construction.
Paper : Learning to build with LEGOA visualization of keypoint detection under image perturbations, motivating robustness verification for perception-driven autonomy.
Paper : How robust is learned perception?Using formal task specifications to organize assembly, allocate work across robot teams, and coordinate building and warehouse services.
Six robots share an office building, coordinating service tasks with hierarchical temporal logic. Multiple tasks progress in parallel while the robots respect ordering, routing, and privacy constraints.
A team with three robot types coordinates item collection and delivery across a warehouse, satisfying collaboration and sequencing requirements.
A grid-world benchmark demonstrates temporal logic task allocation and coordinated motion for a robot team.
Assign robots to the labeled regions to satisfy the task’s group-visit, ordering, and recurring-visit requirements. Plan routes through the grid while avoiding obstacles and collisions. The paper provides the full benchmark specifications.
A pair of robot arms assembles a LEGO model following hierarchical task instructions.
Assemble the LEGO model from the bottom layer upward. Follow the brick-placement instructions within each layer and the precedence constraints between layers, coordinating the two arms.
The research behind the demonstrations.
Browse publications