The comments section also contains some good suggestions. After wiping your brush well on a rag or paper towel, you can use either Goop hand cleaner, baby oil or safflower oil from the grocery store to remove the residual paint. Then soap & water (regular bar soap or Murphy's Oil Soap) will finish the job. Maybe top it off with Master's or Winsor-Newton Brush Cleaner.
There is also a post by Larry Seiler on WetCanvas! where he describes how to melt down Ivory soap. He uses it as an inexpensive alternative for cleaning brushes in the classroom.
Of course, Gamblin Gamsol is a reasonably safe solvent, made from cosmetic grade petroleum distillate with all harmful aromatic solvents refined out of it, so it is minimally toxic whether inhaled, ingested or exposed to skin. It is good for thinning oil paints and mediums as well as cleanup.