Although TaOx memristors have demonstrated encouraging write/erase endurance and nanosecond switching speeds, the linear current-voltage (I-V) characteristic in the low resistance state limits their applications in large passive crossbar arrays. We demonstrate here that a TiO2-x/TaOx oxide heterostructure incorporated into a 50 nm× 50 nm memristor displays a very large nonlinearity such that I(V/2) ≈ I(V)/100 for V ≈ 1 volt, which is caused by current-controlled negative differential resistance in the device.