Former defence secretary Liam Fox warned that Britain was "powerless" to prevent some "very unsavoury individuals" from entering the country under the guise of protecting Russia's sportspersons, according to the Daily Mail.
The comment has sparked a row with Russia, with the Kremlin terming it as "typical paranoia".
"We face a very large number of FSB agents coming to London during the Olympics, including some very unsavoury individuals. There's no way we can do anything about that," Fox was quoted as saying.
The Federal Security Service of the Russian Federation or the Federal'naya Sluzhba Bezopasnosti Rossiyskoy Federatsii (FSB) is the country's main domestic security agency.
"When the Russians decide they have to send people to protect their team, they'll choose a very large number of FSB people to do that. London's going to be an interesting place next summer," he said.
Dmitry Peskov, spokesman for Russian Prime Minister Vladimir Putin, said: "We are not concerned about his opinion. That is typical paranoia."