Sidwell fired Reading into the lead five minutes before the break after rushing on to a Doyle cross to blast the ball into the top corner from eight yards
Sidwell's provider then made it two on 68 minutes with a well-taken shot that left Millwall keeper Colin Doyle stranded.
Millwall's rotten run of fortune continued as Tony Craig was sent off less than a minute into the second half.
The south east Londoners are desperately short of quality and will be hoping new chairman Peter de Savary is as keen to spend money on players in the new year as he was to splash the cash on a deafening pre-match fireworks display.
Reading started slowly and seemed content to play long balls up to the front two, while Millwall played like they had something to prove against the league leaders.
Bobby Convey came close to scoring midway through the half with a 22-yard free kick from just outside the penalty area but the winger's shot was safely caught by the Millwall keeper.
Alan Dunne provided Millwall's first chance when the Lions were down to 10 men after Ben May had come off the field for lengthy treatment due to a head injury.
But Dunne summed up Millwall's season when he was left unmarked only to head against the bar after Josh Simpson had curled in an inch-perfect cross.
The chance sparked the visitors into life and after Sidwell's strike, Convey nearly made it two as he beat Marvin Elliott then Marcus Phillips only to be denied by keeper Doyle.
Craig made it worse for Millwall less than a minute after the second-half restart when he was sent off for felling Doyle in the box.
Millwall keeper Doyle saved the resulting Sidwell penalty and Paul Robinson bravely tackled the ball out of the midfielder's path to deny him the rebound.
The home side immediately went on the attack and Bruce Dyer was denied by Marcus Hahnemann after breaking free of the Reading defence.
The pressure on 10-man Millwall finally told in the 68th minute when Kevin Doyle beat Phillips to chip past his namesake in goal from a tight angle to seal the game.