Ivanović made his first football steps with FK Srem in 2002, following in his father’s footsteps. Here, he made his professional debut and his performances began to attract attention from Serbia’s First League clubs.

In December 2003, Ivanović joined OFK Beograd, in what was then the First League of Serbia and Montenegro. During his time with Belgrade, his team performed above expectations, even reaching the semi-finals of the 2004 UEFA Intertoto Cup. After two years of playing in Serbia’s top division, Ivanović was courting interest from clubs across Europe.

Ivanović’s under-21 debut took place against Macedonia in Ohrid on 15 December 2003 in a match that Serbia and Montenegro ended up winning 4–1. He scored his first goal only two days later against the same opponent in a 7–0 rout.

Ivanović received his first cap on 8 June 2005 in Toronto, Ontario, Canada, when Serbia and Montenegro played a friendly against Italy, coming on as a 77th-minute substitute for his club teammate Marko Baša in an eventual 1–1 draw. He was not included at the 2006 FIFA World Cup and did not play an international again until after the country’s dissolution that year.

In January 2006, Ivanović was signed by Russian Premier League club Lokomotiv Moscow.

He took part in the 2006 U-21 Championships in Portugal, where he made four appearances and scored one goal for Serbia and Montenegro. Ivanović was given the captain’s armband during this time and continued to captain independent Serbia to the 2007 U-21 Championships final where they lost to the host, the Netherlands.

Ivanović scored his first international goal on 12 September 2007 in a UEFA Euro 2008 qualifying match against Portugal in Lisbon; following a Dejan Stanković in-swinging freekick, he scored with two minutes remaining to seal a 1–1 draw.

On 15 January 2008, English Premier League side Chelsea confirmed that the club had agreed terms with Lokomotiv for Ivanović’s transfer, subject to the agreement of personal terms and the passing of a medical, having beaten competition from the likes of Milan, Ajax, Juventus and Internazionale for his signature. Though the transfer fee was undisclosed, it was speculated to be in the £9 million range. Lokomotiv later announced the transfer fee was €13 million (£9.7 million) and according to the club, the transfer was the largest in Russian football history. Ivanović signed a three-and-a-half-year contract with Chelsea the following day, where he was given the number 2 shirt last worn by Glen Johnson.

Chelsea were in a poor run of form, culminating with a 2–0 loss away at Liverpool followed by a goalless home draw versus lowly Hull City, all of which led to Luiz Felipe Scolari’s removal on 9 February. New manager Guus Hiddink continued omitting Ivanović, who was by now completely out of the first-team picture. His first start under Hiddink finally came on 4 April away at Newcastle United, almost two months after the Dutchman’s arrival at Stamford Bridge. It was the player’s first league appearance in more than three months for the club, and it happened only days after he scored the winning goal for the Serbian national team in a 2010 FIFA World Cup qualifier away at Romania. Despite playing very little competitive football over the previous months (his appearances restricted to national team and cup competitions with Chelsea), Ivanović responded with assured defensive displays.

Ivanović began the season at right back but eventually moved back centre back due to injuries to his teammates Alex and John Terry and the return to fitness of first choice right back José Bosingwa. His first goal of the season scored in an away game against Blackburn Rovers in the Premier League on 30 October 2010, a late header that secured Chelsea 2–1 win. He then scored a brace in a 4–1 win in the Champions League against Spartak Moscow on 4 November 2010, netted by a powerful header and a neat shot. His second goal in the Premier League, or fourth in all competitions, came with a header during the 3–1 loss to Arsenal.

Early in 2011, Ivanović scored his first goal of the year on 15 January 2011, in a 2–0 home win in a Premier League match against Blackburn Rovers at Stamford Bridge, with a shot from close range.

On 10 February 2011, Ivanović signed a new five-and-a-half-year-contract with Chelsea, keeping him at the club until the summer of 2016. After signing this new long-term deal, he said, “It is good news for me, it is a very big step for me in my career and I want to try to help Chelsea win a lot more trophies.” He also added, “These things take a little time but my first vision was always to stay with Chelsea. There were not a lot of problems because I wanted to stay, the club wanted me to stay and I want to thank everyone who helped with the deal. I am very happy with that.” He ended the season by being one of three nominations for Chelsea Player of the Year, losing out in the end to goalkeeper Petr Čech.

Ivanović scored the winning goal in the Champions League round of 16 on 14 March 2012 to complete the turn around against which was decisive extra time winner against Napoli, which Chelsea won 4–1 and 5–4 on aggregate at Stamford Bridge. On 31 March 2012, this marked Roberto Di Matteo’s start at Chelsea with an important win. He then went on to help Chelsea overcome Benfica in the quarter-finals and beat favourites Barcelona in the semi-finals, though he received a second yellow in the second leg and so missed the final against Bayern Munich due to suspension. Ivanović scored a double against Aston Villa in a 2–4 away win. Ivanović scored again in the following game against Wigan Athletic in a 2–1 victory, with a controversial offside goal.

Ivanović played all of Chelsea’s pre-season games and captained the side against Seattle Sounders FC, leading the side to 4–2 victory. On 12 August, Ivanović started the Community Shield match against Manchester City and was sent off for a two-footed tackle on City defender Aleksandar Kolarov. He started Chelsea’s Premier League campaign by scoring an early goal against Wigan on 19 August, helping the Blues to a 2–0 opening round win. He scored again off an assist from Eden Hazard, as Chelsea defeated Reading 4–2 on 22 August. The Serbian then went on to score a goal as Chelsea defeated Norwich City 4–1 on 6 October 2012. He was sent off in a Premier League game against Manchester United after a professional foul on Ashley Young, a game Chelsea lost 3–2. He scored his first ever League Cup goal in 1–5 win over Leeds United on 19 December. On 23 December 2012, Ivanović scored his fifth goal of the season in an 8–0 thrashing of Aston Villa.

On 28 February 2012, Ivanović scored on his first appearance as captain, a 2–0 friendly win over Armenia in Cyprus. Under Siniša Mihajlović, he played as a centre-back during several friendly matches before returning to play as a right-back regularly. On 28 December 2013, Ivanović was named Serbian Footballer of the Year becoming the third player to win the award twice (others including Nemanja Vidić and Dejan Stanković) and the first Serbian player ever to win it in consecutive years. Ivanović scored twice for the first time in an international on 7 June 2015, as Serbia defeated Azerbaijan 4–1 in a friendly in Austria.

Ivanović again displayed his recent goal-poaching prowess and opened his goalscoring account for the calendar year on 5 January 2013, adding a third goal to the scoreline in a third round FA Cup victory against Southampton. This was also the Serb’s first ever goal in the FA Cup. In Paolo Di Canio’s debut as Sunderland manager on 7 April, Ivanović divereted David Luiz’s long range shot past the outstretched arms of Black Cats goalkeeper Simon Mignolet, securing a 2–1 victory for Chelsea.

In the Europa League Final on 15 May 2013, Ivanović headed in the winning goal in the third minute of second-half stoppage time to give Chelsea a 2–1 victory over Benfica. The goal was the Serb’s eighth of the campaign in all competitions, and first in the Europa League, and secured a second successive European title for the London club and the 11th major trophy of the Roman Abramovich era. The win also meant that for ten days, the Blues would hold both the Champions League and Europa League titles at the same time, until the final of the 2013 Champions League final on 25 May. Because of Ivanović’s heroic performance at both ends of the pitch, the Serbian defender was named the Man of the Match. Following the match, teammate Frank Lampard lauded Ivanović and described him as a “great man” and added, “You only have to look at him to see what a beast he is, he goes under the radar a bit because he just does his job, but what a player… He has been an absolute professional and he is a man you want on your side.”

Ivanović was subject to an oral assault by Liverpool striker Luis Suárez in their Premier League fixture on 21 April 2013. The game ended in a 2–2 draw after Suárez scored a 97th-minute equaliser to salvage a draw. Replays showed that as the two tussled for an incoming cross from Steven Gerrard, which was deflected out by Ryan Bertrand for a corner, Suárez dug his teeth into the right arm of Ivanović. Suárez was found guilty of violent conduct and was handed a ten-match ban for the assault from the FA, but Ivanović did not press formal charges following the match. He initially did not accept the apology from Suárez but afterwards the defender said he calmed down and accepted it.

Ivanović became an integral part of newly appointed Chelsea manager José Mourinho’s plans as he started almost all games, missing just one league game so far due to an injury he sustained in the 2–1 win against Liverpool at Stamford Bridge. He made his first appearance of the season at Stamford Bridge on 18 August 2013, helping his side keep a clean sheet with a 2–0 defeat of Hull City. He scored his first goal of the season three days later, heading in Chelsea’s winning goal in a 2–1 league defeat of Aston Villa. Ivanović scored the only goal of Chelsea’s away victory over title rivals Manchester City on 3 February 2014, bringing the sides level on points through 24 league matches.

Ivanović made his first appearance of the 2014–15 season on 18 August 2014, scoring Chelsea’s third goal in a 3–1 win over Burnley at Turf Moor. On 30 August, he scored Chelsea’s second goal, after just three minutes, in a 6–3 win over Everton at Goodison Park. Ivanović was sent off in added time on 26 October after a foul on Ángel Di María at Old Trafford in a Premier League game against Manchester United; ending in 1–1 tie directly from the resulting free kick from Di María.

On 27 January 2015, he scored the winning goal, a header from a Willian free kick against Liverpool in the first half of extra time in the League Cup second leg semi-final to help Chelsea win 2–1 on aggregate and reach the final for the seventh time. On 7 February, Ivanović scored Chelsea’s winning goal of a 2–1 win against Aston Villa at Villa Park. On 17 February, he scored the team’s away goal in a 1–1 draw with Paris Saint-Germain in the round of 16 of the Champions League. Four days later, he scored the opening goal of a 1–1 draw with Burnley for his fourth goal in six matches. On 1 March he started – and played the full 90 minutes – in the League Cup final against Tottenham Hotspur, helping his side to a 2–0 win for their first piece of silverware of the season. On 26 April, Ivanović, along with five of his Chelsea teammates, were included in the PFA Premier League Team of the Year, and a week later the team won the league title.

At the beginning of the season, Ivanović was named as the club’s new vice-captain. On 2 August 2015, Ivanović made his first appearance of the season in 1–0 defeat of the FA Community Shield against London rivals Arsenal. Ivanović scored his first goal of the season in the first game after the departure of manager José Mourinho, against Sunderland. On 22 January 2016, Ivanović signed a new one-year contract extension with Chelsea. On 27 February, he scored an 89th minute winning header in a 2–1 win over Southampton. Ivanović made his 350th appearance for Chelsea in all competitions on 1 March against Norwich, becoming the fourth overseas player to do so.

On 1 February 2017, Ivanović joined Russian side Zenit Saint Petersburg on a two-and-a-half-year deal after a successful nine-year spell with Chelsea. On 16 February 2017, Ivanović made his Zenit debut in their 2–0 away defeat to Anderlecht in the Europa League round of 32.

In June 2018, he was included in the final 23-man squad for the 2018 FIFA World Cup. There he appeared on two matches, against Costa Rica and Switzerland, and he became the player with most appearances for the Serbian football team in its history, having played 105 matches.

On 15 September 2020, West Bromwich Albion announced the signing of Ivanović on a 1-year deal.