Recommender systems are a very broad concept, widely applied in everything from advertising algorithms to song recommendations. In currently published literature, there are at least hundreds of millions, if not billions, of papers. So if someone is going to maintain a pile of papers, they might as well apply recommender system ideas to those papers themselves. As far as I know, ResearchGate and Semantic Scholar have both done quite well in this regard, and, incidentally, our AMiner has also done a little bit of work in this area. I happened to be invited to answer this question on "Ha-Zhihu" (sadly not the Nobel-worthy platform), and, trembling with awe, I've prepared the following words as a response, hoping they might help someone who happens to pass by.

Applying recommender systems to the paper domain is, at its core, still a recommender system, so all the general recommender system techniques still apply. But papers themselves give us a lot of more specific features to work with.

First, a brief version of my take:

  1. Papers are entities with relatively standardized structure. Even across different fields, we can always find a title, abstract, keywords, journal/conference, and references, and their semantics are similar. This means we can extract features across many dimensions more easily, and we can tune weights to make the results match our expectations.
  2. Because papers are structured, a lot of external resources built on top of them become usable. For example, many organizations like CCF and Google have manually curated domain-related features. We can easily say: let's run a topic model over our hundred million papers and cluster them into 100 categories — and those 100 categories already have existing datasets, because CCF/Google have already organized them by hand.
  3. Papers are written by people, and most are attached to a journal/conference. We can easily aggregate all the papers and formalize them into graph models like paper–person–paper, or paper–journal/conference–paper, and then run computations on top of those graphs.

I'm just a programmer. Although I was quite pleased to receive a Hire Letter for a Data Scientist position, honestly speaking, my biggest skill is reading other people's papers, writing my own implementation, tweaking things under someone else's model, and joining competitions. Asking me to prove things would be beyond my ability. So the following is just a personal, implementation-oriented perspective on what's different compared to a typical recommender system. If you're not interested, you can leave now.

--------------------------- Divider line with no substance --------------------------

We all know that:

  1. Papers are written by people.
  2. In most cases, a person's research field stays fairly stable.
  3. A person's collaborators' fields also tend to stay fairly stable.
  4. A journal/conference's topic tends to stay fairly stable; even comprehensive journals like Nature or Science have many sub-journals, and each sub-journal's topic stays fairly stable.
  5. Papers cite each other; if paper a cites paper b, then the topics of a and b are mostly similar, if not identical.
  6. Papers have titles, abstracts, and keywords, and the weight of a text term appearing in each of these is obviously different.

Given these premises, when a new user arrives, how can we make recommendations based on their history and their queries?

First, prepare features and candidate content.

First of all, we need to collect papers. We are very poor — we are not data producers, we are data movers. There are several sources of paper data. One is collaboration: someone packs up their data and hands us a copy, or gives us an API key so we can search a bit more than anonymous users — but someone has to pay for that, and the service quality is not to be discussed. The second is that some websites provide dumps. A big thank you to the conscientious DBLP — I'm practically on my knees to them. The third is directly scraping paper profile pages from some websites. There's a pitfall here: we cannot use Tsinghua machines to scrape, because Tsinghua pays them a large amount of "protection fee" every year, so our IP sees slightly more content than the outside world. But if we fire up crawlers, the consequence is that the other side warns the university, the university publicly criticizes our dear boss, and our dear boss grins menacingly at me. So although we could use proxies, to be safe we just rent machines outside to collect the data.

Second, we need to integrate the data. For example, DBLP and ACM paper data overlap heavily, so we need to integrate heterogeneous data and deduplicate. My approach is to keep one copy of the raw records from each data source first, and then create separate tables for the integrated data. First find similar ones by title, then use edit distance to rule out pairs whose titles differ too much, then compare authors — in short, a pile of dirty work. Incidentally, traditional edit distance really is quite "ed" (exhausting); with a long title it struggles to death, so I use Fang Shen's fail-fast improvement, which I particularly love — I've implemented it in every language I know. After that it's just ordinary rule-based comparisons of various kinds.

Third is author name ambiguity (NA): two papers both have an author named Jiawei Han — is it the same person? Based on currently known information, we can only say probably yes or probably no. This is a painful operation; our results suffer from both over-splitting and over-merging — it practically makes me kneel. For instance, I personally know at least five people named Wang Wei (or homophones thereof), all called "Wei Wang" in English papers. I sheepishly admit that to this day we still haven't solved this problem. For a long time, the dialogue between a certain tall, handsome, basketball-playing PhD student in our lab (who also has a girlfriend) and our dear boss went like this: Q: "I think the results still aren't great. Look at so-and-so, clearly wrong." A: "How do you think I'd prefer to die?" (This dialogue is fictional; their discussions are very friendly and harmonious; my dark mind fabricated this to frame them.)

Fourth is preparing various other related features. For example, from a paper's title and abstract we extract keywords using several methods, and we use Wikipedia and Prof. Li Juanzi's semantic tools to hierarchically organize hypernyms/hyponyms for some topics (LDA and such — sorry, I don't know the details), plus assigning domains to journals, conferences, etc. (This part is still very rough; entity binding for conferences themselves is still quite ugly.)

With all the above data as features, we can do some modest recommendation:

  • The first is text search: take the (historical or current) keywords, find papers, rank them, order the tail roughly, and then rerank the top ones with more computationally expensive algorithms.

  • The second is collaborative filtering — just as you'd imagine, with the features mentioned above thrown in.

  • The third is Social Network Analysis. Actually nothing earth-shattering: we find and formalize the social relationships between disambiguated authors on papers. The simple version: authors as nodes, co-authoring a paper as an edge. A more complex version: authors and papers as different node types, with "writes" as the edge type. Even more complex: add journals/conferences into the graph. The more complex it is, the more papers you can publish; the simpler it is, the easier to implement and deploy online to run the whole graph and watch the results. That said, a paper at KDD 2015 used the simplest relationship; the main idea was to apply a random-walk-like approach to the simplest graph described above, predicting other points close to a given point — possibly first-degree relations, possibly second- to third-degree. Since it's random wandering, theoretically it can go as far as the walk length, but in the end the top-n results rarely are. The main achievement is that with this sampling, after tuning parameters, the performance isn't much worse than computing over all nodes and edges, but the speed is an order of magnitude faster, so it can still produce results when the models from related work have already ground to a halt (paper: http://arxiv.org/abs/1504.02577 Demo: https://github.com/yuikns/panther). Of course, it could actually be somewhat better, but you know, my coding skill isn't enough.

[github_widget repo="yuikns/panther"]

And so, recommending papers quietly turns into the pattern of recommending people, then finding their papers, and then ranking.

This article was completed on a phone while foraging for food and getting exercise; since it was inconvenient to search, it obviously contains many errors and lacks detailed sources. I'll try to supplement it after I finish writing the code someday, hoping it helps someone.