I haven't tried Swarm, but to some degree assume it can give the same effects as Docker Compose with several services. I also less sure of the effects if you never have communication between containers, but I think perhaps there may still be the same or similar issue.
What I experienced when doing not exactly a load test, but just processing a large dataset through multiple docker containers started from a docker compose config, was that the default docker network loopback (docker0) was saturated. After creating a docker network that the various nodes were configured to use, things got a lot better.
So this is the question for you, do all the containers in the swarm talk via docker0? If yes, read up on docker networks in relation to swarm in particular.