# How Robot Vision Works How a robot arm's depth and colour cameras find an apple, measure its distance, name it and plan a reach to grab it\. By ycode99 Topic: Computing, AI & Electronics [Open the interactive Jig](https://ask.jig.com/j/VYRqnQvfjdLh) ## Transcript ### 1. A robot that can see This robot arm wants to pick up the apple, but nobody told it where the apple is\. It has to find it with the bar\-shaped depth camera above the bench\. Let’s follow every step between seeing and grabbing\. Drag to look around\. Pinch or scroll to zoom\. ### 2. Light becomes numbers The colour camera splits its view into a grid of tiny squares called pixels\. Each pixel stores three numbers: how much red, green and blue light arrived\. That is about two million pixels, up to 30 times a second\. The grid drawn here is much coarser than the real one\. To the robot, a picture is just a long list of numbers\. ### 3. Two eyes measure distance One flat picture cannot tell a small apple close up from a big apple far away\. So the camera also has two infrared eyes, 5 centimetres apart, like your two eyes\. A near object shifts more between their two views than a far one, and that shift gives its distance\. Hold up a finger and close one eye, then the other\. Your finger jumps; the far wall hardly moves\. ### 4. Invisible dots help A plain, smooth surface looks the same everywhere, so the two eyes struggle to match its points\. The small window in the middle sprays a pattern of infrared dots onto the scene\. The dots give every patch a pattern the eyes can match\. Infrared light is invisible to people; the dots are drawn in red here\. Shiny, see\-through and very dark things still confuse depth cameras\. ### 5. A picture of distance Matching points across the whole view makes a depth map: a distance for every pixel\. Here it is painted warm for near and cool for far\. Cameras like this work best between about 0\.3 and 3 metres away\. At 2 metres the error is usually under 2 percent, about 4 centimetres\. ### 6. What is that? Next, a neural network studies the colour picture\. It learned from many example photos that people had labelled, so it can draw a box around each object and name it\. The number is how sure it is, not a promise\. The robot does not see like we do\. It finds patterns in numbers that match what it learned\. ### 7. Where exactly? A box on a picture is not enough to grab something\. The robot lays the apple’s outline over the depth map to find where it sits in 3D, measured from the camera\. A calibration then turns that into a spot the arm can reach\. Calibration is measuring once exactly where the camera sits compared with the arm\. ### 8. Reach and grab Now the robot works out an angle for every joint so the gripper ends up right at the apple\. Press the button to watch it reach down, grip gently and place the apple in the tray\. Turning a target position into joint angles is called inverse kinematics\. ### 9. Look again After moving, the robot looks again to check that the apple really landed in the tray\. Seeing, deciding and moving in a fast loop is how robots in warehouses, farms and homes work safely around us\. The same steps help robot vacuums avoid socks and self\-driving cars spot people crossing\.