Understand how remote desktop connection works; A user logs in and is in effect, virtually "in front of the computer" as if they were there. When another user connects, they TOO are virtually "in front of the computer".
You say you have up to 7 users on the computer running programs. When this happens, each user is running their own instance of the program. This tends to get very RAM hungry and can bring your machine to it's knees as far as performance is concerned.
Since you seem to have plenty of disk space, I would suggest adding memory (all 8 GB if you can, or ramp up the RAM over time to 8GB as money allows) to increase your performance under a heavy load. 1GB under Win Server 2003 is really the bare minimum if the machine is to be used for more than just file sharing or printing.
I hope this helps
If you are satisfied, please check yes above to accept this answer. Thank you very much.
As always bonuses are always welcomed and highly appreciated.
1 Other Expert Agrees with this!