the fuel pump does not come on when the key is on, but I can hear the pump on for a sec when I let off the starter with the key. I already replaced the pump, the old 1 worked when I put volts to it. I have fuel to the fuel rail when I checked it but could have been after a start attempt. So why doesn't the pump come on unless I'm trying to start it????

 

I would guess there is still enough pressure in the rail for the pump to not come on until some of it is let off. Have you pressure tested the system?