Depends on the foam and the finish used (if you don't want to take the paint off)
If using a water based glue then use a damp paper towel or similar.
Whatever solvent is in the glue is what you use. For Epoxy use denatured alcohol (and get it before the epoxy starts to stiffen)
For general cleaning of the models I use foaming bathroom cleaner... after testing on scrap foam... since the bubbling action can lift dirt out of the little crevices between beads of EPO or EPS. I haven't had much luck cleaning EPP.