QU at TREC-2015: Building Real-Time Systems for Tweet Filtering and Question Answering

Abstract

This paper presents our participation in the microblog and LiveQA tracks in TREC-2015. Both tracks required building a "real-time" system that monitors a data stream and responds to users' information needs in real-time. For the microblog track, given a set of users' interest profiles, we developed two online filtering systems that recommend "relevant" and "novel" tweets from a tweet stream for each profile. Both systems simulate real scenarios: filtered tweets are sent as push notifications on a mobile phone or as a periodic email digest. We study the effect of using a static versus dynamic relevance thresholds to control the relevancy of filtered output to interest profiles. We also experiment with different profile expansion strategies that account for potential topic drift. Our results show that the baseline run of the push notifications scenario that uses a static threshold with light profile expansion achieved the best results. Similarly, in the email digest scenario, the baseline run that used a shorter representation of the interest profiles without any expansion was the best run. For the LiveQA track, the system was required to answer a stream of around 1000 real-time questions from Yahoo! Answers. We adopted a very simple approach that searched an archived Yahoo! Answers QA dataset for similar questions to the asked ones and retrieved back their answers.