Discussion

Left ArrowBack to discussions page
alanjonesalanjones Posts: 10 Apprentice
Hi guys, Im wondering Is it possible to perform object detection using ur10 and ros ?
If so, could any of you provide an example of the source code ?
Would appreciate if any of you can update me asap, thanks!

Comments

  • matthewd92matthewd92 Posts: 702Founding Pro, Tactile Sensor Beta Testers Handy
    What kind of object detection are you trying to do? Using a force sensor, vision system, something else?
  • jprobergejproberge Posts: 29Tactile Sensor Beta Testers Crew
    Hi @alanjones !

    Are you trying to use Robotiq's wrist camera to perform object detection on ROS, or is it another vision system? If using Robotiq's wrist camera, you'll be able to acquire frames from it on ROS (without the possibility of controlling the flash leds though), but you would have to code the object detection algorithm by yourself, since we did not make any package performing object detection automatically yet. While there might not be an official package provided by Robotiq at the moment, maybe it is possible to find another publicly-available project from a third party, but I'm unfortunately not aware of such project at the moment.

    Please feel free to post more questions below!

    Best regards,

    Jean-Philippe Roberge, 
    ROS Package Developper / Maintainer 
    [email protected]

  • alanjonesalanjones Posts: 10 Apprentice
    @matthewd92 @jproberge ;I am thinking of using the wrist camera to perform object detection via ros.
    I may need to use an additional software to detect regular shaped objects so if you guys have any recommendation please feel free to suggest !
    May I ask is it possible to use opencv code on ur robot and ros ?
    Also, as I'm entirely new to ros I would appreciate if step by step guidance can be provided to guide me through the installation process ( laptop is running on windows 10 ) , thanks !

  • jprobergejproberge Posts: 29Tactile Sensor Beta Testers Crew
    Hi @alanjones ,

    If you plan to use the wrist camera to detect regular-shaped objects, OpenCV is indeed a very good starting point and will most likely contain all the tools you'll need to perform this task. OpenCV can totally work with ROS, since it is a library, thus after it has been installed, you can import it in your projects whenever you need it. Also, while Robotiq doesn't officially support using the wrist camera on ROS, it is still possible to use a package such as the typical usb_cam ros package to acquire frames from the camera. On the other hand, please note that it will not be possible to control the flash leds on the wrist camera using ros, since there is no API to perform such thing at the moment.

    If you're entirely new to ROS, then no worries, there are a lot of tutorials to help you discover how great it is :) However, ROS runs on Linux, and although it is now possible to install it on Windows 10 the vast majority of ROS users still use Linux. Also, if you take a look at that last link I've put, you'll see that the ROS version running on Windows 10 is not a native windows 10 version of ROS, it's rather a Ubuntu version of ROS running on-top of the Windows Subsystem for Linux. Hence, I strongly suggest you use ROS on Linux, you'll have way better support, there's also a way bigger community of developers. That doesn't mean you should throw away your Windows 10 OS, but you could rather install Linux on another partition and dual boot. 

    If you want a very high level, step-by-step procedure of what I'd probably do to use your laptop and a wrist camera to perform object detection, well here it is. Please note that this is just to give you a very broad and general overview of what you could do to make things work, but in reality, the required steps may differ.

    1) Backup your computer, as a wrong operation here could affect your partition(s) and data located on it (them).
    2) Re-partition your laptop drive: Shrink your Windows 10 partition, create two new partitions (ext4 and swap space). Google "Install Linux alongside Windows 10" if you never did that and take your time and do everything carefully.
    3) Install a Linux version of your choice. As a new ROS user, I would choose Ubuntu 16.04, which is a LTS version (LTS = Long Term Support). You could also install Ubuntu 18.04, but if you are new to this Linux world, then staying one LTS version behind the latest LTS release will do no harm, plus you'll also have more support and see more posts related to that version.
    4) Once Linux is installed and dual-boot works properly on your machine, then you can go ahead and install ros, I would suggest to install ros-kinetic, for the exact same reason given in 3). This procedure is pretty straightforward and you should be just fine by following all these steps.
    5) Create your catkin workspace and learn how to compile it by doing tutorials #1.1 and #1.3 from these official ros tutorials. I highly recommend doing also, at least 1.5, 1.6 and 1.11 (if you plan to write your code in C++) or 1.12 (if you'd rather use Python).
    6) Go to our official github repo and download (or git clone) the ros-kinetic-devel branch of that repo and put it in the source folder of your catkin workspace. Check the Robotiq ros wiki page to make sure to install the dependencies. Compile your catkin workspace now containing the Robotiq ros package stack.
    7) Install and compile all other needed packages (e.g.: your robot manufacturer's ros package, etc...)
    8) Create your own package and start writing your first ros node. Import OpenCV, subscribe to the proper usb_cam's topic to acquire a frame, treat it with your algorithms (here is a good starting point on how to do this).

    This may seems scary at first, especially if you are new to all this, and it is true that there is a more or less considerable learning curve, but in the end, you'll see that you'll benefit from the time you "invested" to learn ROS. Also, feel free to add more questions below!

    Best regards,

    Jean-Philippe Roberge, 
    ROS Package Developper / Maintainer 
    [email protected]
  • alanjonesalanjones Posts: 10 Apprentice
    @jproberge Hi , thanks for the prompt reply!
    May I know instead of wrist camera can I use vision ( webcam ) to detect object and for opencv can I use any programming language ? 
    Since I'm still new to ros, what programming language will you suggest me to use for coding?
    I'ld prefer something simple to learn so that I can start coding.
  • jprobergejproberge Posts: 29Tactile Sensor Beta Testers Crew
    Hi @alanjones ,

    You could indeed use a webcam instead of the wrist camera, and still be able to treat the frames with ROS. I actually did that a few times and it works flawlessly. Also, for open CV the primary interface is in C++, but there are bindings also in Java and Python, and actually there is also an old, but still working and extensive interface in C.

    You can use ROS with C++ and Python, most of its users use one or both language(s). For the sake of completeness, note that it also works with LISP and they have experimental libraries in Java and Lua. Anyway, most of the time, developers will go for either Python and/or C++.

    Here is a non-rigorous list of things to consider if you have to learn such a new language for your project:

    C++: It's fast! It is a compiled language so it will run way faster than Python most of the time. So if speed is a concern for you, this is clearly something to be aware of. On the other hand, it is object-oriented and more complicated to learn than Python.

    Python: Easy to learn! One of the easiest language to learn and it is so intuitive. It is not compiled, it is rather an interpreted language so it will run slower than C++ in general, for most applications.

    Summary: Choose Python or C++ depending on your context, then use that language for ROS and OpenCV, as they both support the two languages :) 

    Good luck, and feel free to add more questions below.

    Best regards,

    Jean-Philippe Roberge, 
    ROS Package Developper / Maintainer 
    [email protected]
  • alanjonesalanjones Posts: 10 Apprentice
    @jproberge Thank you so much for the information ! 
    I have some more questions below and I hope you'll be able to help me out by answering them , thanks ! (: 
    May I know if I were to install simulator how would the procedure be like ? 
    I did some research online and found out that people use gazebo for simulation and is it the only simulation software available ? 
    For simulator can i get it to work from home without having physical connection to the robot and will it work on the actual robot if I were to test out the program ?
    As for the package , is it a must for us to create our own package in order to write or edit code ? 
    How can I know if I have already created my catkin workspace and what is the function of it ? 
    Do you think is it easier to work with object detection using opencv or cad model ? 

    Sorry for bombarding you with so many questions !
  • jprobergejproberge Posts: 29Tactile Sensor Beta Testers Crew
    edited December 13
    @alanjones : you're welcome, and here are my answers to your latest questions, hope it helps:)

    1) May I know if I were to install simulator how would the procedure be like ? 
    Gazebo is the most used simulator with ROS. If you installed the full version of ros, using, for example "sudo apt-get install ros-kinetic-desktop-full", then it should already be installed on your machine. You can try to launch gazebo simply by typing "gazebo" in a terminal and pressing enter. If it is not installed, then you can still manually install it by following the simple steps from this tutorial.

    2) I did some research online and found out that people use gazebo for simulation and is it the only simulation software available ? 
    Gazebo is the most used simulator among ros users and it works very well with ROS. That's the one most people use. However, there are other simulators as well depending on what you want to do. See MuJoCo or Pybullet for example...

    3) For simulator can i get it to work from home without having physical connection to the robot and will it work on the actual robot if I were to test out the program ?
    Sure, that's the whole point of a simulator: not to be dependent of the hardware. So yes, you can definitely work from home in simulation without any hardware around. Then, it "should" work on the real robot as well as it was working in simulation if you setup everything properly.

    4) As for the package , is it a must for us to create our own package in order to write or edit code ? 
    Well, it depends on what you want to do, maybe for a very simple procedure it would be fine just to use the command line, but for most applications, yes you should write your own package. If your goal is to perform object detection and to make the robot pick up a part that was detected, then I would definitely recommend to create your own package. I would also strongly suggest not to modify any code that was provided by a manufacturer / ros-industrial consortium for a specific device. You should most of the time create your own package, which is not a harder thing to do. See for instance this tutorial as well as this one.

    5) How can I know if I have already created my catkin workspace and what is the function of it ? 
    Have you followed the tutorial about installing and configuring your ros environment? I strongly suggest to do it if you haven't done it yet. The catkin workspace is the folder where catkin_make will "make" your ros executables from your source code. It is an organized folder ramification where your include files / source files / message definitions / service definitions / compiled programs are. You already created your catkin workspace if you followed the aforementioned tutorial and/or if you typed something like "mkdir -p catkin_ws/src" followed by, "cd catkin_ws/src" and "catkin_init_workspace". In summary, you already created your catkin workspace if you created a folder with a "src" subfolder, where you used "catkin_init_workspace" command to initialize your catkin workspace.

    6) Do you think is it easier to work with object detection using opencv or cad model ? 
    I'm not sure what you mean by "cad model" in the context of object detection? Using Opencv is probably the most easy way to rapidly be able to detect object online with a webcam or with Robotiq wrist camera, so I would definitely recommend to explore this library. However, if you explained a bit more what you meant by using the "cad model" for object detection, I might be able to provide a more accurate answer. Were you suggesting to use a known cad model of a part that has to be detected by your algorithm? If so then OpenCV can also help with that.

    Again, don't hesitate to contact me if you have more questions!

    All the best,


    Jean-Philippe Roberge, 
    ROS Package Developper / Maintainer 
    [email protected] 
  • alanjonesalanjones Posts: 10 Apprentice
    Hi @jproberge , yes I have gone through some ros tutorials and installed the necessary stuff on ros.
    May I know after I've clone/download universal robot package from github, should I open up each folders one by one and copy the code into ros or is there a proper way to install the files into ros?
    Would appreciate if you can provide me with some guidance on that (:
    As for this question 'Were you suggesting to use a known cad model of a part that has to be detected by your algorithm?', I'm planning to use something that is available on grabcad or any other platforms to test out in gazebo!
  • jprobergejproberge Posts: 29Tactile Sensor Beta Testers Crew
    edited December 15
    Hi @alanjones

    Nope, you just have to copy the whole package stack to your src/ folder an compile it using catkin_make, not individual folders. 

    Best regards,

    Jean-Philippe Roberge, 
    ROS Package Developper / Maintainer 
    [email protected] 
Sign In or Register to comment.
Left ArrowBack to discussions page