If your hard drive fails on your linux server it can always be stressful, but that is why you should always have some kind of raid setup and backups available. If you do not have a hardware raid card in your server you can use software based raid. I always set my servers up with RAID 1 (mirrored disk) and backups to an external server. Remember RAID is not a backup solution since deleting a file on your mirror will remove it from both disks. Below are the basic steps for replacing a mirrored disk in software raid using mdadm.
Remove all raid partitions on the failed disk
mdadm --fail /dev/md0 /dev/sdb1
mdadm --fail /dev/md1 /dev/sdb2
mdadm --remove /dev/md0 /dev/sdb1
mdadm --remove /dev/md1 /dev/sdb2