List All Postgres related packages

dpkg -l | grep postgres

Remove all above listed

sudo apt-get --purge remove postgresql postgresql-8.3  postgresql-client  postgresql-client-8.3 postgresql-client-common postgresql-common  postgresql-contrib postgresql-contrib-8.3

Remove the following folders

sudo rm -rf /var/lib/postgresql/
sudo rm -rf /var/log/postgresql/
sudo rm -rf /etc/postgresql/

Remove the postgres user:

sudo deluser postgres