TDSM 12.1
From The Data Science Design Manual Wikia
Parallel processing has tightly-coupled computation, and the tasks are CPU-bound. It happens on one machine.
Distributed processing has loosely-coupled computation, and the tasks are memory/IO-bound. It involves sharing resources across multiple machines.
