Xbox Live has a minimum speed restriction built-in.
directly from XBOX DASHBOARD:
"This test confirms your network has a sufficient Maximum Transmission Unit (MTU). MTU is a setting on your router or gateway that can affect network performance. Xbox Live requires a minimum MTU setting of 1364. If your console does not pass this test, you will not be able to sign in to Xbox Live. See your router or gateway instruction manual for information about changing the MTU setting."
basically 512K or better connection.
aleast due to the design of the XBOX Live network, and API it means that the only people who experience lag and poor gameplay as a result of a bad connection are those who have the bad connection.
Well unless they're hosting the game, then everyones screwed; they shout abuse at the host and leave (or hope he leaves allowing the connection to jump to someone with a better connection)