We currently are using a 2 finger robotiq gripper with a UR5 and a CB2 controller. Is there a way to do nested subroutines with the robot using the pre-programmed gripper subroutines in conditional statements. We are running to a programming error stating the robot is incapable of calling upon a subroutine within a subroutine. 

The UR is not capable of calling subroutines within a subroutine.  What version software are you running on the robot?  I thought they had made a change to this but not sure.  You can call the script functions directly and honestly, its faster anyways to do that.

For instance if you want to open the gripper you can call rq_open() or rq_open_and_wait() or you can call rq_move(xx) or rq_move_and_wait(XX) where xx is the amount you want opened between 0 and 255.  That will allow you to control the amount of opening.  

You can use subroutines but you have to trick the program.

On the highest level in the program, make an if, loop or switch statement and call a subroutine in it. 
then copy the if, loop or switch statement into the part of the program where you need the subroutine. 
Now the call subroutine is not refused and works fine.