Director: Prof. Matthew Lease
Research Areas (specific): Information Retrieval (IR) •
Crowdsourcing & Human Computation (HCOMP) •
Natural Language Processing (NLP)
Research Areas (general): Artificial Intelligence (AI) • Human-Computer Interation (HCI)
Overview: IR is the science behind search engines such as Google, Bing, & Yahoo. Crowdsourcing and human computation engage online workers to train or augment automated artificial intelligence algorithms. My IR research seeks to improve core search algorithms, reliably evaluate search systems, and to enable new forms of search. My HCOMP research seeks to optimize crowdsourced data collection (e.g., quality, cost, and speed), to expand the reach of crowdsourcing to tackle new problems, and to investigate broader socio-technical questions of how paid crowdsourcing is transforming digital work and the lives of workers engaged in it. At the intersection of IR and HComp, I develop crowdsourcing methods to better scale IR evaluation methodology while preserving its reliability. Both IR and HComp place people at the center of computing: system users in IR and online workers in HComp. I thus seek to orchestrate effective man-machine partnerships which creatively blend front-end HCI design with back-end AI modeling of people and their tasks. By capitalizing on the respective strengths of each party - man and machine - we can compensate for the other's limitations to create a whole greater than the sum of its parts. For example, IR systems can utilize front-end HCI design to empower searcher intuition and creativity, while back-end AI algorithms interpret ambiguous human queries, sift through vast information, and suggest potentially relevant results. In HCOMP, front-end HCI design can enable workers to more easily understand and complete tasks, while back-end AI modeling of workers and tasks enables principled optimization of data collection.
Selected Research & Demos
Mucahid Kutlu, Tyler McDonnell, Tamer Elsayed, and Matthew Lease. Annotator Rationales for Labeling Tasks in Crowdsourcing. Journal of Artificial Intelligence Research (JAIR), 69:143--189, 2020. Award Winning Papers Track. [ bib | pdf | blog-post | data | conference-website ]
Soumyajit Gupta, Mucahid Kutlu, Vivek Khetan, and Matthew Lease. Correlation, Prediction and Ranking of Evaluation Metrics in Information Retrieval. In Proceedings of the 41st European Conference on Information Retrieval (ECIR), pages 636--651, 2019. Best Student Paper award. [ news | bib | pdf | data | sourcecode | slides | tech-report ]
Kezban Dilek Onal, Ye Zhang, Ismail Sengor Altingovde, Md Mustafizur Rahman, Pinar Karagoz, Alex Braylan, Brandon Dang, Heng-Lu Chang, Henna Kim, Quinten McNamara, Aaron Angert, Edward Banner, Vivek Khetan, Tyler McDonnell, An Thanh Nguyen, Dan Xu, Byron C. Wallace, Maarten de Rijke, and Matthew Lease. Neural Information Retrieval: At the End of the Early Years. Information Retrieval, 21(2-3):111--182, 2018. [ bib | pdf | slides | tech-report ]
An Thanh Nguyen, Aditya Kharosekar, Aditya Kharosekar, Saumyaa Krishnan,
Siddhesh Krishnan, Elizabeth Tate, Byron C. Wallace, and Matthew Lease.
Believe it or not: Designing a Human-AI Partnership for
In Proceedings of the 31st ACM User Interface Software and
Technology Symposium (UIST), pages 189--199, 2018.
[ bib |
Brandon Dang, Martin J. Riedl, and Matthew Lease. But Who Protects the Moderators? The Case of Crowdsourced Image Moderation. In 6th AAAI Conference on Human Computation and Crowdsourcing (HCOMP): Works-in-Progress Track, 2018. [ bib | pdf | demo | blog-post | slides ]
Tyler McDonnell, Matthew Lease, Mucahid Kutlu, and Tamer Elsayed. Why Is That Relevant? Collecting Annotator Rationales for Relevance Judgments. In Proceedings of the 4th AAAI Conference on Human Computation and Crowdsourcing (HCOMP), pages 139--148, 2016. Best Paper Award. [ news | bib | pdf | blog-post | data | slides ]
Hohyon Ryu, Matthew Lease, and Nicholas Woodward. Finding and Exploring Memes in Social Media. In Proceedings of the 23rd ACM Conference on Hypertext and Social Media, pages 295--304. ACM, 2012. [ bib | pdf | demo | sourcecode | video | Amazon award ]
Lu Guo and Matthew Lease. Personalizing Local Search with Twitter. In Workshop on Enriching Information Retrieval (ENIR) at the 34th Annual ACM SIGIR Conference, 2011. [ bib | pdf | sourcecode | video ]
Current PhD Studets (as of Fall 2020)
Join our Lab!
High School Students: apply for UT's Summer Research Academy