If the toilet is still rocking, it could be from debris under it, loose pipes or a broken toilet flange. To remove the toilet: Turn off the water supply. Flush the toilet to empty the tank and ...