QArm buzz-wire sim

4-DOF arm guiding a 30 mm loop along a 3D wire. Contact closes the circuit.

Controller

Tracking noise

Training

Learning on

Turn off to test the current policy with no exploration.

Learning curve

episodes

–

success, last 20

1.00

ε explore

env steps

grad updates

–

TD loss

finished wiretouchedsuccess rate

Q-values right now

Each cell is a push direction for the loop (centre = no push). Brighter = higher predicted future reward. The outlined cell is the action taken.

Joint angles

J1 base yaw	0.0°
J2 shoulder	0.0°
J3 elbow	0.0°
J4 wrist	0.0°

Only 4 joints, so the loop can't always tilt to match the wire. The sim cheats here; the real arm can't.

How the DQN is set up

State (8)	offset, drift, bend now, bend ahead
Actions (9)	push −/0/+ on 2 axes
Reward	+1 − 0.6·offset² /step
Touch / finish	−20 / +10, ends episode
Network	8 → 64 → 64 → 9, ReLU
Training	Double DQN, Adam 5e-4, γ 0.97
Replay / batch	30k / 32, target sync 500

The agent only controls the loop's sideways correction as it is carried along the wire; the arm joints follow via IK. Training runs live in your browser.