If the driveshaft carrier is gone - I assume you mean the rubber piece - then I would address that immediately. Since you must drop the driveshaft to fix the carrier, check all the u-joints to make sure they move freely with no play nor binding. You probably need to replace the carrier bearing when you replace the rubber piece.

Also, when you split the drive shaft, make sure you mark the front and rear sections as the shaft is balanced as one piece.