Order preparation with Robotiq Wrist Camera

bcastets

I would like to share with you a method to do order preparation with Robotiq Wrist Camera.

I this time the COVID is still impacting our everyday live and the need on contact limitation if pushing people to place online orders.

The processing of those orders require a lot of manipulation on a large variety of products. The operation have to find each items of the order and place it in the order box.

There is way to manage this repetitive task with a cobot equipped with Robotiq Wrist Camera and a gripper (ePick, airpick, Hand-e, 2F85/140,...).

The main challenge of order preparation application to be have to identify and manipulate a large variety of products.

In a previous post I present a technique with the Robotiq Wrist Camera to pick object by detecting a hidden background rather than the object itself:

https://dof.robotiq.com/discussion/2208/robotiq-wrist-camera-detect-an-unlimited-number-of-object-types-without-any-object-teaching#latest

Using this same technique, it is possible to detect the presence of an object in a storage box and process it. This approach simplify the process because there is no need to teach any objects. No need to use 3D camera or AI complex technology.

Here below is the picture of the type of storage box used for this demonstration.

The inside of the box is a cardboard matrix where are placed products. A circle is draw at the bottom of each placement. Using the Robotiq Wrist Camera, the robot search for placement where the background circle is hidden and dive on it to pick the object.

On the side of the box is an image on the product inside the box. The Robotiq Wrist camera use this side image to identify box contents and organization (product inside the box, pattern of the inside cardboard matrix,...). In this example I use an image but using a barcode would be more appreciate. The good thing is that the Robotiq Wrist Camera can read barcode !

The following video present this order preparation concept. The first part of the video shows an order processing using Robotiq ePick vacuum gripper, the second part show an alternative using Robotiq Hand-e gripper.

https://www.youtube.com/watch?v=RAv65xby7qs

I hope this video will give you some idea to have your cobot do order preparation for you using the Robotiq Wrist Camera.

This solution have the advantage to be simple and affordable.

I attach program and printed references used in this demo.

bcastets

I had few people trying to replicate this order preparation concept with the Robotiq wrist camera but they face some difficulties with waypoint setup and especially the setup of the palletizing node which is not made to be use this way.
Also my first program works with item name "ePick" and "camera" which is a bit confusion. On this second program I use generic item name such as "item1", "item2",...

In order to help people replicate this concept rewrite the program and removed the palletizing node.
The program is not made to be use as it is. You have to understand it and adjust it to fit your requirements.
Even if this version of the program is more accessible it is still advance usage of the camera. Beginners may have some difficulties to understand it.

Here below is a description of the concept.

Problem

• Order preparation can involve a large variety of items which have to be picked form storage boxs. Detecting of objects in storage box is difficult. Teaching objects to a vision system is not really possible because of the variety of items and variety of orientation in storage boxs.

Solution

• Instead of detection the items themselves, the camera search background circle marks placed on each object placement of the box cardboard matrix. If a background circle is visible it means that there is not item in the placement. If it is not visible, an item is present and can be picked by the robot.
• The robot can validate box contents by reading a bare code on the side of the box.
• Each item box have its own cardboard matrix pattern. The robot adjust the program to match the carboard matrix pattern of the item.

Limitations

• Background marks have to be place at the bottom of each box. It could be done with a paper placed at the bottom
• The robot is doing blind picking. It dive on item until sensing its surface and then activate the gripper. In this case the gripper is ePick but it could also be a Hand-e gripper for example. In some case blind picking may not be efficient
• Items have to be stored in boxes segmented by a cardboard matrix.

Possible extension

Adjust the program to manage different size of box.Adjust the program to load item information from an external item databaseAdjust the project to receive order from an external source (network)Store box pattern information in box bare code
Here below is a description of the program (For details please check comments in the program):

In the before start program is imported some script functions and set some variables and settings used in the program.
Box Origin, Box size, card board matrix pattern and number of ordered item are defined.

2 types of items are processed in this program. A third item is suppressed so that people understand how they could had some more items in the program.



Imported script functions are:

• Modulo(z, mod)
• pointNearPointInList(point, pointList, distance)
• boxMatrixPoints(boxOrigin, boxXsize, xSectionsCount, boxYsize, ySectionsCount)

Her below is a drawing illustrating orientation for boxMatrixPoints(boxOrigin, boxXsize, xSectionsCount, boxYsize, ySectionsCount) function.



Here below is the main program:

1. The robot request the box it needs to process the order which have been set in the before program section.
2. Using the camera the robot check if a box is present. To do that the robot chcek if it can see the big circle mark which should be hidden by the box.
3. The robot verify the item available in the box by reading bar code on the side on the box.
   
4. Using the camera the robot check if items are present in cardboard matrix placements and pick the requested quantity of items. To do that the robot checks on which placement small background circles cannot be seen.
5. The program stop once all items of the order have been picked.

Here below is the subprogram used to check where items are present in the card board matrix.

bcastets

Here are program files:
https://share.robotiq.com/public/5773d36c5169

bcastets

Here is a video presenting how to setup the program. I had some problem with the sound which cut some time. Sorry for that !


https://www.youtube.com/watch?v=Q0nZM5mcbKc