Koloff debuted in the World Wide Wrestling Federation in late 1969, managed by "Captain" Lou Albano. He soon started a rivalry with then-WWWF World Heavyweight Champion Bruno Sammartino. On January 18, 1971, Koloff defeated Sammartino in Madison Square Garden for the WWWF World Heavyweight Championship by pinfall after a knee drop from the top rope, ending Sammartino's seven and two-third years reign. Koloff lost the championship twenty one days later to Pedro Morales, essentially being used as a "transitional champion". After the loss, Koloff remained a contender for the title but never reclaimed it, leaving the WWWF in 1971.
Koloff eventually joined World Championship Wrestling (WCW) alongside Nikita Koloff as The Russians. The Russians (which also included Krusher Kruschev) were a top villainous group from 1984 to 1986. After Nikita turned on Ivan to join their enemy Dusty Rhodes in 1986, the latter teamed with Vladimir Petrov and Dick Murdoch to get revenge. Koloff's biggest WCW feuds were against Rhodes, The Road Warriors, The Rock 'n' Roll Express (Ricky Morton and Robert Gibson) and Magnum T.A..
In 1988, Koloff spent time in Paul Jones' faction, The Paul Jones Army, where he was a "coach" of sorts for The Powers of Pain (The Barbarian and The Warlord). He later split with Jones, reunited with Nikita, and feuded with Jones' team, The Russian Assassins, before leaving WCW in January 1989.
He spent time in Extreme Championship Wrestling, appearing on the very first ECW card in 1992. He defeated The Sandman on ECW Hardcore TV on May 15, 1993.
On February 18, 2017, Koloff passed away.