Alexis Sanchez scored his first Barclays Premier League goal only for Ulloa to instantly level in the first half.
But Ulloa missed a glorious chance to win it after the break when he fired wide with just Wojciech Szczesny to beat.
Jamie Vardy was also denied by the goalkeeper as the hosts refused to be fodder for the big guns.
And, despite Sanchez's goal, the game underlined Arsenal's desperate need for striking reinforcements as Yaya Sanogo struggled to make an impact.
Sanogo was given the responsibility to lead Arsenal's attack after Olivier Giroud was ruled out until January with a broken tibia.
And boss Arsene Wenger, who left Jack Wilshere on the bench, was urged to sign a striker before Monday's transfer deadline by the travelling fans.
Leicester were unchanged from their 2-0 defeat at Chelsea, reverting back to full strength following Tuesday's 1-0 Capital One Cup defeat to Shrewsbury.
Former Inter Milan midfielder Esteban Cambiasso was among the substitutes after completing his free transfer on Thursday.
And he would have been impressed with Leicester's bright start as they took the game to the Gunners.
Arsenal suffered an early blow when Laurent Koscielny clashed heads with Jeffrey Schlupp, forcing the defender off for treatment
And while the Gunners were down to 10 men, Leicester carved out their best chance.
Riyad Mahrez swapped passes with David Nugent but his 25-yard effort skidded just past the post.
It was a sign of the hosts' intent but the Foxes' were undone on 20 minutes when Sanchez, making his first league start, pounced.
Neat play saw Santi Cazorla loft the ball to Sanogo and, when the striker made a hash of his effort, the ball fell to Sanchez to smash in from six yards.
The lead lasted just two minutes though as Per Mertesacker allowed Schlupp to run and his cross was crashed in by Ulloa from 12-yards.
The striker beat Koscielny to the ball with the Frenchman still feeling the after-effects of his knock and he was immediately replaced by Calum Chambers.
The goal was all Leicester deserved for their tenacious start and while they lacked the Gunners' style their efforts unsettled their opponents.
Sanogo struggled against the hosts' backline with the Foxes closing ranks to deny Cazorla and Mesut Ozil space but with Sanogo floundering there was little focal point.
Nugent and Ulloa were busy for Leicester though and Chambers was almost caught out by Nugent in stoppage time but wriggled free before Mahrez fired at Szczesny.
Ulloa continued to threaten after the break and should have fired Leicester ahead on 53 minutes.
Schlupp's ball dissected Arsenal and when Chambers dived in the striker could only hit the side netting from eight yards.
Arsenal immediately broke and Aaron Ramsey headed straight at Kasper Schmeichel with Sanogo waiting for a tap in before the striker himself was thwarted by Schmeichel's smart save.
Leicester's threat began to wane as the Gunners dominated possession without breaching the organised Foxes.
Lukas Podolski and Alex Oxlade-Chamberlain replaced Cazorla and Sanogo but the Foxes remained defiant and almost snatched it on 84 minutes.
Substitute Vardy broke from a corner and swapped passes with Nugent, only to be denied by Szczesny.
Source : PA