Jonny Gomes opened the 12th with a double off Rafael Perez (0-1) and scored on Guzman's hit to center. Guzman had been called up from Triple-A Durham before the game.
Scott Dohmann (2-0) threw two scoreless innings for the victory.
The AL Central-leading Indians had won three in a row following a four-game losing streak. Cleveland has a 1 1/2 game lead over second-place Detroit, which lost 9-3 to the New York Yankees.
B.J. Upton and Carlos Pena homered for the Devil Rays, who are 14-36 since June 25. Tampa Bay had not hit a home run in its previous eight games, a club record.
Victor Martinez hit a tiebreaking sacrifice fly in the 10th to give the Indians a 3-2 lead. The Devil Rays pulled even in the bottom half on a two-out RBI single by Carl Crawford off Joe Borowski, who blew his fifth save in 39 opportunties.
Indians starter C.C. Sabathia gave up two runs and five hits over eight innings. The left-hander had six strikeouts, giving him 1,101 in his career — the eighth Cleveland pitcher to reach at least 1,100 strikeouts.
Tampa Bay's James Shields allowed two runs and six hits in six innings. He struck out six and walked one.
Shields was hit on the lower right leg by Grady Sizemore's liner in the fourth, but remained in the game after taking a couple warmup tosses.
Martinez put the Indians ahead 1-0 on an RBI single in the first. He has 65 RBIs in 68 games when batting fourth in the lineup.
Jhonny Peralta made it 2-0 with a fourth-inning sacrifice fly.
Sabathia retired his first 11 batters before Upton's homer on a 3-2 pitch with two outs in the fourth, cutting the Tampa Bay deficit to 2-1. The shot ended Tampa Bay's homer drought at 291 at-bats.
Upton also made a nice running catch on Sizemore's drive to the center field warning track in the eighth. He also took an extra-base hit away from Ryan Garko with another running grab for the first out in the 11th.
Pena tied it at 2 on his 27th homer of the season in the seventh, equaling a career high set with Detroit in 2004. It was his first homer in 14 games.
Notes:@ The Indians are off Monday before starting a three-game series Tuesday at Detroit. ... Sabathia moved past Mike Garcia (1,095) into eighth place on the Indians career strikeout list. ... Tampa Bay placed SS Ben Zobrist (strained right side muscle) on the 15-day disabled list. ... Pena has hit 20 homers against Cleveland since 2002. Only Paul Konerko of the Chicago White Sox (21) has more.