So I got Oracle 11gR2 installed on my headless Ubuntu 14.04 LTS machine and noticed Application Express was version 4.0.2.
Oracle’s instructions for upgrading seemed straight forward so I ran with it. I received no errors but was unable to access /apex/apex_admin. I came across this post which uses a different approach to loading the images so I gave that a shot. Since 11g was a fresh install with nothing to loose I wiped everything and started from scratch. Unfortunately I found myself in the same position, unable to access /apex/apex_admin.
After some searching, I discovered it was as simple as allowing external access.
After executing the following command, I was up and running with Application Express 5.0.3