As a ten-year-old, Petit competed in the 1994 Little League World Series, representing the championship-winning Coquivacoa Little League team. This was the first time a Venezuelan team had won the LLWS title in eight appearances.

Petit signed with the New York Mets as an international free agent on November 15, 2001. He made his professional debut in 2002 with the Venezuelan Summer League where he went 3–5 with 2.43 earned run average (ERA) in 12 games, 11 starts.

In 2003, Petit split the season between the Rookie-Level Kingsport Mets and the Short-Season Brooklyn Cyclones. He finished third in the Appalachian League in strikeouts, fourth in strikeouts per nine innings and fifth in ERA. Petit also placed second in Appalachian League in fewest bases on balls per nine innings and allowed the second fewest runners per nine innings. He was named Pitcher of the Week for the week of July 14 to July 20. He was promoted to Brooklyn of the New York–Penn League on August 23. He recorded 20 strikeouts in 12⁄3 innings with the Cyclones.

Petit spent time with Binghamton and the Triple-A Norfolk Tides in 2005. He went 9–3 with 2.91 ERA in 21 starts at Double-A, then went 0–3 with 9.20 ERA in three starts with Norfolk after being promoted on August 22. He made four starts for the Norfolk Tides, including one in the playoffs against the Toledo Mud Hens. He lost all three regular-season starts, allowing 16 runs on 24 hits in 14⁄3 innings. He did however earn a win in the playoff start against Toledo, allowing three earned runs on six hits in eight innings, recording 14 strikeouts. He was named to the Eastern League’s mid-season All-Star team and finished second in the league in ERA. He again pitched in the Venezuelan Winter League, going 5–1 with one save and 2.01 ERA in nine games.

In November 2005, the Mets traded Petit along with Mike Jacobs and Grant Psomas for Carlos Delgado. He appeared in 15 games, one start, with the Marlins, going 1–1 with 9.57 ERA during his three major league stints. He pitched primarily out of bullpen, going 0–1 with 10.18 ERA in 14 outings as a reliever. He made his major league debut on May 14 against the Pittsburgh Pirates, allowing one hit with two strikeouts in one inning. He held the Atlanta Braves scoreless in three innings during second appearance on May 17 at Turner Field, recording a career-high three strikeouts. Petit suffered first major league loss on May 19 against the Tampa Bay Devil Rays, allowing a 10th-inning walk-off home run to Aubrey Huff. He was optioned to the Triple-A Albuquerque Isotopes on June 18 after going 0–1 with 7.36 ERA in seven outings. He was recalled on July 4, and went 1–0 with a 10.57 ERA in two games before he was optioned back to Albuquerque on July 17. He was again recalled on September 1 going 0–0 with 11.74 ERA in final six appearances of season. He made his first major league start and earned first major league win on July 5 against the Washington Nationals.

Petit was traded to the Arizona Diamondbacks on March 26, 2007, for Jorge Julio. He picked up his first win for the Triple-A Tucson Sidewinders on April 15 against the Colorado Springs Sky Sox, tossing six innings, allowing two earned runs off six hits with five strikeouts. He was recalled on April 22 and made his Diamondbacks debut that night against the San Francisco Giants, allowing two runs over seven innings in the loss. He earned wins in five of his next six starts, going 5–0 with a 2.21 ERA. He earned his first win for the Diamondbacks on July 3 against the St. Louis Cardinals, allowing one run over 5⁄3 innings.

Going into the 2009 season, Petit was the likely candidate for the fifth starter role in the pitching rotation or a long-relief role, out of the bullpen. He started the season in the major leagues for the first time in his career and posted a 0–1 mark with a 4.70 ERA in six relief appearances before being optioned to Tucson on April 27. He made 11 starts for the Sidewinders, going 3–3 with a 4.80 ERA. Petit was recalled on June 27 and remained in the Majors for the remainder of the season. He made his first start of the season for the D-backs on July 2 and earned a no-decision in a 4–3 loss against the Milwaukee Brewers after allowing only one run on two hits with four strikeouts in six innings. He held opposing batters to a .216 batting average for the season, including a .213 average as a starter. In 2009, Petit was mainly a spot starter for the Diamondbacks rotation and went 3–5 with a 4.31 ERA in 19 games, of which eight were starts.

On August 4, 2009, Petit took a no-hitter into the eighth inning against the Pittsburgh Pirates before it was broken up in the eighth with a single by Ronny Cedeño. It was the only hit he allowed in eight innings. He finished the season 3–10 with a 5.82 ERA in 23 games, of which 17 were starts.

Petit was claimed off waivers by the Seattle Mariners on November 5, 2009. He was designated for assignment on February 6, 2010. On February 9, Petit cleared waivers and was sent to Triple-A. On March 17, he was released by the team. However, he re–signed to a minor league deal 10 days later.

In 2011, Petit played for the Oaxaca Warriors of the Mexican League.

Petit began 2013 with Triple-A Fresno, but on July 23, he was called up to help the pitching staff during a double-header. He pitched in the first game of the double-header after Eric Surkamp had a poor start. In 5.1 innings, he struck out 7 and gave up 2 runs. On July 28, he was designated for assignment. Petit subsequently cleared waivers and was sent back to Fresno. He was recalled by the Giants on August 23, as a starter.

On September 6, 2013, making just his third major-league start of the year since joining the rotation as an injury replacement, Petit came within one strike of pitching a perfect game against his former team, the Arizona Diamondbacks. It was broken up on a single by pinch hitter Eric Chavez on a 3-2 count with two outs in the ninth inning. Petit would get the following out, finishing the game with 7 strikeouts and no walks on 95 pitches. The Giants won that game 3-0. The game was Petit’s first career complete game and shutout. He was the 12th pitcher in MLB history to lose a perfect game with two outs in the 9th inning.

During the 2014 season, Petit worked mostly in relief, with occasional starts. Making a spot start in place of the injured Matt Cain on July 22 and giving up five runs (increasing his season ERA to 4.11), Petit recorded the last out of the fifth inning and subsequently proceeded to post six consecutive perfect relief appearances, of lengths varying from 1 to 4 ⁄3 innings, totalling 38 consecutive retired batters. On August 28, Petit returned to the Giants starting rotation in place of Tim Lincecum and set down the first eight Colorado Rockies to set a new MLB record for consecutive batters retired at 46 (over a period of eight games), breaking the record of 45 formerly held by Mark Buehrle. Buehrle’s streak included his perfect game and the starts before and after. Buehrle had broken a 1972 record of 41 consecutive retired batters that had been set by Jim Barr over the course of two complete-game wins, from the third inning of one to the seventh inning of the next; Barr’s mark had been tied in 2007 by Bobby Jenks over the course of 14 relief appearances. He also broke Buehrle’s record for consecutive perfect innings pitched by 0.1 with 15.1. After giving up a third-inning double to opposing pitcher Jordan Lyles to snap the streak, Petit proceeded to complete six innings, allowing four hits and a run, to record the win and reduce his season ERA to 3.44.

Petit reached free agency after the 2015 season. The Washington Nationals announced on December 14, 2015, that he accepted a one-year deal with an option for a second year, which was declined. On April 7, against the Miami Marlins, he made his Nationals debut.

On February 8, 2017, the Los Angeles Angels signed Petit to a minor league contract with an invitation to spring training. On March 30, he was added to the opening day roster. On April 5, he made his Angels debut against the Oakland Athletics.

On December 7, 2017, Petit signed a two-year contract with the Oakland Athletics which includes a club option for 2020. Petit finished his first season with the A’s with a 7-3 record and an even 3.00 ERA over 93 innings. He continued his success the following season, registering an ERA of 2.71 in 80 games, prompting the Athletics to exercise their option on him.

On October 29, 2018, he was selected as one of the MLB All-Stars during the 2018 MLB Japan All-Star Series