Energy-time entangled photons are critical in many quantum optical phenomena and have emerged as important elements in quantum information protocols. Entanglement in this degree of freedom often manifests itself on ultrafast time scales making it very difficult to detect, whether one employs direct or interferometric techniques, as photon-counting detectors have insufficient time resolution. We implement ultrafast photon counters based on nonlinear interactions and strong femtosecond laser pulses to probe energy-time entanglement in this important regime. Using this technique and single-photon spectrometers, we characterize all the spectral and temporal correlations of two entangled photons with femtosecond resolution. This enables us to witness energy-time entanglement using uncertainty relations and to directly observe nonlocal dispersion cancellation on ultrafast time scales. These techniques are essential to understand and control the energy-time degree of freedom of light for ultrafast quantum optics.