As of version 4.0 a simple ajax performance tracker is introduced. This tracker will measure the run time and the peak performance of the search function.
This is a great tool for comparing performances when changing configuration or enabling/disabling plugins.
The tracker can only measure the run time of the search function.
This is different from the whole ajax request run time. Before that WordPress initializes, loads all of the plugins, executes all the tasks needed and then executes the search function.
If the tracked run time is low yet the ajax response feels much slower, it means that something else is clogging up the ajax requests. It's usually a plugin (or more plugins) or the theme.
In this case, the plugin causing the issue should be disabled. It will slow down not only the ajax search plugin, but all plugins using ajax requests.
To investigate which plugin might cause the issue, try to disable the one-by-one until the front end ajax request feels faster.