The way that it is handled is multiple CPU modes of operation, normally "supervisor" and "user".
The CPU processes instructions differently when in the different modes, the lower the privilege level the less instructions can be executed. When the CPU attempts to execute an instruction that the current privilege mode does not have access to it throws an exception that an upper layer catches. This upper layer is normally the kernel, but can be other things depending on virtualization extensions and other low level details. From there whatever handles the exception will either terminate the process that executed it, crash or make the process think that it worked.
It varies from architecture to architecture, but that is basically how it works.