Hi @alanjones !

Are you trying to use Robotiq's wrist camera to perform object detection on ROS, or is it another vision system? If using Robotiq's wrist camera, you'll be able to acquire frames from it on ROS (without the possibility of controlling the flash leds though), but you would have to code the object detection algorithm by yourself, since we did not make any package performing object detection automatically yet. While there might not be an official package provided by Robotiq at the moment, maybe it is possible to find another publicly-available project from a third party, but I'm unfortunately not aware of such project at the moment.

Please feel free to post more questions below!

Best regards,

Jean-Philippe Roberge, 
ROS Package Developper / Maintainer 
j-p.roberge@robotiq.com

Hi @alanjones ,

If you plan to use the wrist camera to detect regular-shaped objects, OpenCV is indeed a very good starting point and will most likely contain all the tools you'll need to perform this task. OpenCV can totally work with ROS, since it is a library, thus after it has been installed, you can import it in your projects whenever you need it. Also, while Robotiq doesn't officially support using the wrist camera on ROS, it is still possible to use a package such as the typical usb_cam ros package to acquire frames from the camera. On the other hand, please note that it will not be possible to control the flash leds on the wrist camera using ros, since there is no API to perform such thing at the moment.

If you're entirely new to ROS, then no worries, there are a lot of tutorials to help you discover how great it is :) However, ROS runs on Linux, and although it is now possible to install it on Windows 10 the vast majority of ROS users still use Linux. Also, if you take a look at that last link I've put, you'll see that the ROS version running on Windows 10 is not a native windows 10 version of ROS, it's rather a Ubuntu version of ROS running on-top of the Windows Subsystem for Linux. Hence, I strongly suggest you use ROS on Linux, you'll have way better support, there's also a way bigger community of developers. That doesn't mean you should throw away your Windows 10 OS, but you could rather install Linux on another partition and dual boot. 

If you want a very high level, step-by-step procedure of what I'd probably do to use your laptop and a wrist camera to perform object detection, well here it is. Please note that this is just to give you a very broad and general overview of what you could do to make things work, but in reality, the required steps may differ.

1) Backup your computer, as a wrong operation here could affect your partition(s) and data located on it (them).
2) Re-partition your laptop drive: Shrink your Windows 10 partition, create two new partitions (ext4 and swap space). Google "Install Linux alongside Windows 10" if you never did that and take your time and do everything carefully.
3) Install a Linux version of your choice. As a new ROS user, I would choose Ubuntu 16.04, which is a LTS version (LTS = Long Term Support). You could also install Ubuntu 18.04, but if you are new to this Linux world, then staying one LTS version behind the latest LTS release will do no harm, plus you'll also have more support and see more posts related to that version.
4) Once Linux is installed and dual-boot works properly on your machine, then you can go ahead and install ros, I would suggest to install ros-kinetic, for the exact same reason given in 3). This procedure is pretty straightforward and you should be just fine by following all these steps.
5) Create your catkin workspace and learn how to compile it by doing tutorials #1.1 and #1.3 from these official ros tutorials. I highly recommend doing also, at least 1.5, 1.6 and 1.11 (if you plan to write your code in C++) or 1.12 (if you'd rather use Python).
6) Go to our official github repo and download (or git clone) the ros-kinetic-devel branch of that repo and put it in the source folder of your catkin workspace. Check the Robotiq ros wiki page to make sure to install the dependencies. Compile your catkin workspace now containing the Robotiq ros package stack.
7) Install and compile all other needed packages (e.g.: your robot manufacturer's ros package, etc...)
8) Create your own package and start writing your first ros node. Import OpenCV, subscribe to the proper usb_cam's topic to acquire a frame, treat it with your algorithms (here is a good starting point on how to do this).

This may seems scary at first, especially if you are new to all this, and it is true that there is a more or less considerable learning curve, but in the end, you'll see that you'll benefit from the time you "invested" to learn ROS. Also, feel free to add more questions below!

Best regards,

Jean-Philippe Roberge, 
ROS Package Developper / Maintainer 
j-p.roberge@robotiq.com

Hi @alanjones ,

You could indeed use a webcam instead of the wrist camera, and still be able to treat the frames with ROS. I actually did that a few times and it works flawlessly. Also, for open CV the primary interface is in C++, but there are bindings also in Java and Python, and actually there is also an old, but still working and extensive interface in C.

You can use ROS with C++ and Python, most of its users use one or both language(s). For the sake of completeness, note that it also works with LISP and they have experimental libraries in Java and Lua. Anyway, most of the time, developers will go for either Python and/or C++.

Here is a non-rigorous list of things to consider if you have to learn such a new language for your project:

C++: It's fast! It is a compiled language so it will run way faster than Python most of the time. So if speed is a concern for you, this is clearly something to be aware of. On the other hand, it is object-oriented and more complicated to learn than Python.

Python: Easy to learn! One of the easiest language to learn and it is so intuitive. It is not compiled, it is rather an interpreted language so it will run slower than C++ in general, for most applications.

Summary: Choose Python or C++ depending on your context, then use that language for ROS and OpenCV, as they both support the two languages :) 

Good luck, and feel free to add more questions below.

Best regards,

Jean-Philippe Roberge, 
ROS Package Developper / Maintainer 
j-p.roberge@robotiq.com

@alanjones : you're welcome, and here are my answers to your latest questions, hope it helps:)

1) May I know if I were to install simulator how would the procedure be like ? 
Gazebo is the most used simulator with ROS. If you installed the full version of ros, using, for example "sudo apt-get install ros-kinetic-desktop-full", then it should already be installed on your machine. You can try to launch gazebo simply by typing "gazebo" in a terminal and pressing enter. If it is not installed, then you can still manually install it by following the simple steps from this tutorial.

2) I did some research online and found out that people use gazebo for simulation and is it the only simulation software available ? 
Gazebo is the most used simulator among ros users and it works very well with ROS. That's the one most people use. However, there are other simulators as well depending on what you want to do. See MuJoCo or Pybullet for example...

3) For simulator can i get it to work from home without having physical connection to the robot and will it work on the actual robot if I were to test out the program ?
Sure, that's the whole point of a simulator: not to be dependent of the hardware. So yes, you can definitely work from home in simulation without any hardware around. Then, it "should" work on the real robot as well as it was working in simulation if you setup everything properly.

4) As for the package , is it a must for us to create our own package in order to write or edit code ? 
Well, it depends on what you want to do, maybe for a very simple procedure it would be fine just to use the command line, but for most applications, yes you should write your own package. If your goal is to perform object detection and to make the robot pick up a part that was detected, then I would definitely recommend to create your own package. I would also strongly suggest not to modify any code that was provided by a manufacturer / ros-industrial consortium for a specific device. You should most of the time create your own package, which is not a harder thing to do. See for instance this tutorial as well as this one.

5) How can I know if I have already created my catkin workspace and what is the function of it ? 
Have you followed the tutorial about installing and configuring your ros environment? I strongly suggest to do it if you haven't done it yet. The catkin workspace is the folder where catkin_make will "make" your ros executables from your source code. It is an organized folder ramification where your include files / source files / message definitions / service definitions / compiled programs are. You already created your catkin workspace if you followed the aforementioned tutorial and/or if you typed something like "mkdir -p catkin_ws/src" followed by, "cd catkin_ws/src" and "catkin_init_workspace". In summary, you already created your catkin workspace if you created a folder with a "src" subfolder, where you used "catkin_init_workspace" command to initialize your catkin workspace.

6) Do you think is it easier to work with object detection using opencv or cad model ? 
I'm not sure what you mean by "cad model" in the context of object detection? Using Opencv is probably the most easy way to rapidly be able to detect object online with a webcam or with Robotiq wrist camera, so I would definitely recommend to explore this library. However, if you explained a bit more what you meant by using the "cad model" for object detection, I might be able to provide a more accurate answer. Were you suggesting to use a known cad model of a part that has to be detected by your algorithm? If so then OpenCV can also help with that.

Again, don't hesitate to contact me if you have more questions!

All the best,


Jean-Philippe Roberge, 
ROS Package Developper / Maintainer 
j-p.roberge@robotiq.com 

Hi @alanjones

Nope, you just have to copy the whole package stack to your src/ folder an compile it using catkin_make, not individual folders. 

Best regards,

Jean-Philippe Roberge, 
ROS Package Developper / Maintainer 
j-p.roberge@robotiq.com 

@jproberge Hi, sorry forgot to add in this link to my error as well. 

Hello @alanjones !

I don't know why you have a compilation error when you tried to compile OpenCv from source, I've taken a look at your files: a segmentation fault error often means there was a bad memory allocation, but it's hard to tell for sure what is the specific cause of your problem or why this is happening on your machine. However, instead of trying to compile from source, why not considering to install the debian package? Have you already installed ros? If so you could simply install OpenCV by typing "sudo apt-get install ros-kinetic-opencv3" (given that the ros version you chose to install is Kinetic). Check out the specific documentation here.

Also it doesn't look like there is an error in the print screen you added in your very last message... it just says everything is already installed.

Finally, when I asked:
' Were you suggesting to use a known cad model of a part that has to be detected by your algorithm? If so then OpenCV can also help with that. ', I was implying that you could use known features from a cad model to detect/recognize a part in an image. There are many tools in OpenCV to help you with that. If the image is 2D then you could simply use moments to try to correlate a cad model with segmented objects in an image, but there are also more sophisticated tools that are available in the OpenCV library. Disclaimer: I'm not even close of being an OpenCV or Vision expert at all, I just worked on a few vision projects in the last years :)

Don't hesitate if there's anything more I can do!

Best regards,

Jean-Philippe Roberge, 
ROS Package Developper / Maintainer 
j-p.roberge@robotiq.com 

Hi @jproberge , thanks for the great help ! 
I did that, added swap space and it worked (:
Please bear with me as I will be asking a bit more questions in this comment!
May I know how should I test the camera / webcam using ros and opencv?
My opencv version is 2.4.9.1 and I tried a few of the youtube tutorials but doesn't work.
These are the tutorials I've tried:
https://www.youtube.com/watch?v=HqNJbx0uAv0  ( encountered error when I tried to perform catkin_make. Here's the link to view my error and other files )
https://www.youtube.com/watch?v=TzKwej7c0fw  (error encountered when I tried to perform npm install electron)
Thanks for the recommendation but I think I may need to detect a 3D image of flat surface / boxes. If that's the case what will you recommend?  :)
Does robotiq has its own tutorials on how to use ur robots and two-finger gripper to perform simulation in gazebo?
I did a quick search and seems like the way to do it is through MoveIt.
Is there a way to use opencv concurrently with gazebo when I want to view the movement of the arm and the object being detected?
Are there any tutorials that you would recommend for opencv?
I'm running short of time so have to work on everything simultaneously.
Hope I'm not scaring you with my questions and have a good day!