Wenger, officially appointed Arsenal boss 20 years and one day ago, had cut a frustrated figure as his side failed to capitalise from numerous goalscoring chances before Koscielny appeared to turn the ball home with his arm in the second minute of added time.
The Frenchman was lurking at the back post and was hit by Alex Oxlade-Chamberlain's close-range shot following a corner.
Burnley had defended brilliantly despite Arsenal's dithering in and around the penalty area and they went close to breaking the deadlock when Michael Keane's second-half header hit the crossbar, but the Gunners extended their unbeaten run in all competitions to nine matches
Arsenal almost made a flying start as Alexis Sanchez's low angled drive from just inside the area deflected off Theo Walcott and spun inches wide
The visitors took the game to Burnley, playing with all the confidence and cohesion you would expect from a side which had won six of their previous seven games in all competitions, and another low angled effort, this time from Walcott, flashed across the face of goal
George Boyd fired wide in the 15th minute after being set up by Jeff Hendrick, but Burnley were forced to play on the counter for spells in the first period such was Arsenal's mastery of possession
The Gunners had little to show for their slick play in the final third, though, as home centre-backs Ben Mee and Keane stood firm, while Burnley striker Sam Vokes headed wide from right-back Matt Lowton's curling cross shortly before the interval
Tom Heaton was forced into the first save of the match in the 47th minute from Sanchez's low drive from inside the area, but Walcott blazed over and Alex Iwobi's mishit shot from a promising position inside the area flew out for a Burnley throw
Mesut Ozil dithered in the area and his shot was blocked before Burnley showed Arsenal how it should be done when going close in the 60th minute, Johann Berg Gudmundsson's header from Stephen Ward's first-time cross turned away at full stretch by Petr Cech
Arsenal fans appealed for a penalty when the ball appeared to ricochet off Keane's arm for a corner but referee Craig Pawson was unmoved before Sanchez showed fine technique to volley just wide after the ball had spun high to him inside the area
Burnley served warning they could easily take all three points when Keane's powerful header from Gumundsson's 75th-minute corner came back off the crossbar, although replays appeared to show Cech got his hand to it
Walcott, hoping to win a recall for England when Gareth Southgate names his full squad after the match, curled a fine shot just beyond the far post in the 85th minute with Heaton beaten
But in the second minute of time added on, Walcott headed on Ozil's corner and when Oxlade-Chamberlain scrambled the ball towards goal at the far post, the ball appeared to hit Koscielny's elbow before it crossed the line to snatch Arsenal their seventh win in eight matches
Source : PA