As Jiggah says it sits in program files when you install it so it will always be active from there.
On the other hand "install for all users" usually just means the applications data is stored somewhere (application data or local settings\application data usually) in C:\Documents and Settings\All Users (adjust for your install drive and OS accordingly) rather than the regular user directory and likewise for the desktop/start menu entries.
It is not terribly advisable as it might have hardcoded directories in there somewhere but you can try moving it from all users to one user and deleting startup entries accordingly.
This is not always done though and some programs will want to write things to their install directory or odd parts of the registry rather than documents and settings (such things were one of the reasons UAC in vista and 7 was very annoying at first and one of the reasons some programs will only work on admin accounts). Depending on the person you are speaking to this is bad coding practice (which is usually combined with other bad coding) but that is a different debate for another time.
You can also prevent programs from running by various means although if the other user is to remain an admin then it can get tricky to truly lock them out.
http://technet.microsoft.com/en-us/library/bb457006.aspx
Failing that you can always do something like make a
http://www.truecrypt.org/ drive/folder/section and install to that or the easier option of
http://portableapps.com/ or maybe a virtual machine.
Most protections are aimed at keeping people from reading the documents of other users rather than programs though.
