Learning to rank is concerned with a model or a function for ranking objects which is useful for various applications like document retrieval and collaborative filtering. The paper introduces two probability model: permutation probability and top one probability.
In document retrieval, there is a collection of documents. Given a query, the ranking function assigns a score to each document, and ranks the document in descending order of the scores. The ranking order represents the relative relevance of documents w.r.t. to the query.
While training, a number of queries are provided; each query is associated with a perfect ranking list of documents; a ranking function is then created using the training data, such that the model can precisely predict the ranking lists in the training data.
To calculate the listwise loss function, both the scores of documents assigned by a ranking function and the explicit or implicit judgments of the documents given by humans are transformed into the probability distribution. Then an metric between the probability distribution can be used as the loss function.
The major contribution of the paper are:
In pairwise approach, the learning task is formalized as the classification of object pairs into two categories (correctly ranked and incorrectly ranked).
The top one probability of on object represents the probability of its being ranked on the top, given the scores of all the objects.
The paper developed a new learning method for optimizing the listwise loss function based on top one probability, with neural network as model and gradient descend as optimization algorithm called ListNet.