To be exact, VVT helps a lot in the lower rev range as the 20v was originally designed to have poor low rev torque. With VVT, it improves driveability and saves fuel as well due to increased intake timing when VVT kicks in. Unless if you're using hicams over 272 duration, there's no need to remove VVT unless you totally cant take the "diesel noise" anymore =)
Its best to retain VVT since one of the main feature of 20V is its VVT and Toyota spent tons of money doing R&D for the 20V. Luckily, for us in malaysia we get the choice to get it serviced professionally. Just so you know, VVT pulley is a wear & tear part which needs to be replaced over time & mileage.