He got his start as a volunteer at Dodge City CC in 1992 before coaching the offensive line at Alabama A&M, South Carolina State, James Madison, Cornell and then Virginia from 2001-02.
Prince took over as the Cavaliers' offensive coordinator from 2003-05, and then succeeded Bill Snyder as Kansas State's head coach from 2006-08. He was fired from that position after going 17-20 in three seasons.
Prince, who spent three years in the NFL before becoming the Scarlet Knights' offensive coordinator last season, is the fourth different person to leave as Rutgers' offensive coordinator after just one year on the job.
Prince was an assistant with the Jacksonville Jaguars in 2012 and with the Indianapolis Colts with Caldwell in 2010 and 2011.
Rutgers is expected to launch a search for his successor immediately.
Under Prince, Rutgers averaged 26.5 points per game and 365.2 yards -- but the unit tailed off over the final seven games of the season, with starting quarterback Gary Nova eventually getting benched for the final three games.