Can I use the same microfiber towel on glass, paint and wheels?

Most experienced detailers recommend dedicating towels to specific tasks, separate cloths for glass, paint, interior and wheels, to avoid cross-contaminating delicate surfaces with grit or chemicals. Color-coded multi-packs make this easier, which is why several picks here are sold in large assorted sets.