a little structure into your session fun:
having four players on stage, drums plus three, pull three session cards.
every non drummer get's one, lay them open.
get the last digit from the birthdate of every player, this is the number of beats in two bars.
tell/show them everyone.
first let the chords/tones ring
enter some rhythm
everyone is allowed to take up every part.
be aware of tonal preferences: after a longer conversation with a trumpet player my session cards mostly do b-flat things. this fits to a former experience, when pulling things with guitar players, one gets a lot of sharps.
usage of two bar structures is recommended for rhythms (clave style), four bar for harmonic rhythms.