Chien-Ju Ho |
My research broadly connects to the fields of machine learning, optimization, behavioral sciences, and algorithmic economics. I am interested in investigating the interactions between humans and AI, including enabling AI algorithms to learn from humans (e.g., in the context of crowdsourcing) and designing AI algorithms to assist human decision-making (e.g., through information design and environment design).
For more information, please see my CV and Google Scholar page. You can also reach me via [chienju.ho at wustl dot edu].
- Lauren's work on investigating the consequences of AI training on human decision making is accepted at the Proceedings of the National Academy of Sciences (PNAS)!
- Received a research award from OpenAI (on leveraging weak human supervision to improve strong ML models) and a seed grant from TRIADS (on investigating human trust in ML models in the domain of housing). Thanks for the support!
- Wei Tang will be joining Chinese University of Hong Kong (CUHK) School of Business as an assistant professor in Fall 2024. Congrats Wei!
Current students:
- Saumik Narayanan (01/2021-present, CSE)
- Lauren Treiman (09/2022-present, DCDS, co-advised with Wouter Kool)
- Alex DiChristofano (01/2023-present, DCDS, co-advised with Patrick Fowler)
- Robert Kasumba (01/2023-present, DCDS, co-advised with Dennis Barbour)
- Tory Farmer (01/2024-present, CSE)
- Wei Tang (01/2018-08/2022, CSE Ph.D.)
- Guanghui Yu (01/2020-08/2024, CSE Ph.D.)
-
In most cases, my collaborators and I choose to have students as first authors and then determine the authorship of non-student authors alphabetically.
-
Research Statement: Behavior-Informed Machine Learning
Chien-Ju Ho
Research Statement, 2023 -
The Consequences of AI Training on Human Decision Making
Lauren Treiman, Chien-Ju Ho*, Wouter Kool* Proceedings of the National Academy of Sciences (PNAS), 2024 -
Adaptive Resource Allocation to Improve Cohort Representativeness in Participatory Biomedical Datasets
Victor Borza, Andrew Estornell, Ellen Wright Clayton, Chien-Ju Ho, Russell L. Rothman, Yevgeniy Vorobeychik, Bradley A. Malin American Medical Informatics Association Annual Symposium, 2024 -
Performative Prediction with Bandit Feedback: Learning through Reparameterization
Yatong Chen, Wei Tang, Chien-Ju Ho, and Yang Liu
In the 41st International Conference on Machine Learning (ICML), 2024 -
The Impact of Features Used by Algorithms on Perceptions of Fairness
Andrew Estornell, Tina Zhang, Sanmay Das, Chien-Ju Ho, Brendan Juba, and Yevgeniy Vorobeychik
In the 33rd International Joint Conference on Artificial Intelligence (IJCAI), 2024 -
Examining the Effects of Explainable Hints in AI-Driven Training (Extended Abstract)
Torrence Farmer and Chien-Ju Ho
In the ACM Collective Intelligence Conference (CI), 2024. -
Rationality-Robust Information Design: Bayesian Persuasion
under Quantal Response
(α-β) Yiding Feng, Chien-Ju Ho, and Wei Tang
In the ACM-SIAM Symposium on Discrete Algorithms (SODA), 2024 - Encoding Human Behavior in Information Design through Deep Learning
Guanghui Yu, Wei Tang, Saumik Narayanan, and Chien-Ju Ho
In the 37th Conference on Neural Information Processing Systems (NeurIPS), 2023 -
Humans Forgo Reward to Instill Fairness into AI
Lauren Treiman, Chien-Ju Ho, and Wouter Kool
In the 11th AAAI Conference on Human Computation and Crowdsourcing (HCOMP), 2023 -
How Does Value Similarity Affect Human Reliance in AI-Assisted Ethical Decision Making?
Saumik Narayanan, Guanghui Yu, Chien-Ju Ho, and Ming Yin
In the 6th AAAI/ACM Conference on AI, Ethics, and Society (AIES), 2023 -
Competitive Information Design for Pandora's Box
(α-β) Bolin Ding, Yiding Feng, Chien-Ju Ho, Wei Tang, and Haifeng Xu
In the ACM-SIAM Symposium on Discrete Algorithms (SODA), 2023 -
Environment Design for Biased Decision Makers
Guanghui Yu and Chien-Ju Ho
In the 31st International Joint Conference on Artificial Intelligence (IJCAI), 2022 -
How Does Predictive Information Affect Human Ethical Preferences?
Saumik Narayanan, Guanghui Yu, Wei Tang, Chien-Ju Ho, and Ming Yin
In the 5th AAAI/ACM Conference on AI, Ethics, and Society (AIES), 2022 - The Influences of Task Design on Crowdsourced Judgement: A Case Study of Recidivism Risk Evaluation
Xiaoni Duan, Chien-Ju Ho, and Ming Yin
In the Web Conference (WWW), 2022 - Bandit Learning with Delayed Impact of Actions
Wei Tang, Chien-Ju Ho, and Yang Liu
In the 35th Conference on Neural Information Processing Systems (NeurIPS), 2021 - On the Bayesian Rational Assumption in Information Design
Wei Tang and Chien-Ju Ho
In the 9th AAAI Conference on Human Computation and Crowdsourcing (HCOMP), 2021
Best Paper Honorable Mention
- Designing and Optimizing Cognitive Debiasing Strategies for Crowdsourcing Annotation (Position Paper)
Chien-Ju Ho and Ming Yin
In the CSCW Workshop on Investigating and Mitigating Biases in Crowdsourced Data (BCD), 2021
- Linear Models are Robust Optimal Under Strategic Behavior
Wei Tang, Chien-Ju Ho, and Yang Liu
In the 24th International Conference on Artificial Intelligence and Statistics (AISTATS), 2021 - Efficient Nonmyopic Online Allocation of Scarce Reusable Resources
Zehao Dong, Sanmay Das, Patrick Fowler, and Chien-Ju Ho
In the 20th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2021 - Optimal Query Complexity of Secure Stochastic Convex Optimization
Wei Tang, Chien-Ju Ho, and Yang Liu
In the 34th Conference on Neural Information Processing Systems (NeurIPS), 2020 - Does Exposure to Diverse Perspectives Mitigate Biases in Crowdwork? An Explorative Study (Short Paper)
Xiaoni Duan, Chien-Ju Ho, and Ming Yin
In the 8th AAAI Conference on Human Computation and Crowdsourcing (HCOMP), 2020 - Differentially Private Contextual Dynamic Pricing
Wei Tang, Chien-Ju Ho, and Yang Liu
In the 19th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2020 - Incorporating Compatible Pairs in Kidney Exchange: A Dynamic Weighted Matching Model
Zhuoshu Li, Kelsey Lieberman, William Macke, Sofia Carrillo, Chien-Ju Ho, Jason Wellen, and Sanmay Das
In the 20th ACM conference on Economics and Computation (EC), 2019 - Leveraging Peer Communication to Enhance Crowdsourcing
Wei Tang, Chien-Ju Ho, and Ming Yin
In the Web Conference (WWW), 2019 - Bandit Learning with Biased Human Feedback
Wei Tang and Chien-Ju Ho
In the 18th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2019 - Incentivizing High Quality User Contributions: New Arm Generation in Bandit Learning
Yang Liu and Chien-Ju Ho
In the 32nd AAAI Conference on Artificial Intelligence (AAAI), 2018 - Eliciting Categorical Data for Optimal Aggregation
Chien-Ju Ho, Rafael Frongillo, and Yiling Chen
In the 30th Annual Conference on Neural Information Processing Systems (NIPS), 2016 - Adaptive Contract Design for Crowdsourcing Markets:
Bandit Algorithms for Repeated Principal-Agent Problems
Chien-Ju Ho, Aleksandrs Slivkins, and Jennifer Wortman Vaughan
Journal of Artificial Intelligence Research, Volume 55, pages 317-359, 2016
(Supersedes the EC'14 paper) - Low-Cost Learning via Active Data Procurement
Jacob Abernethy, Yiling Chen, Chien-Ju Ho, and Bo Waggoner
In the 16th ACM Conference on Economics and Computation (EC), 2015 - Incentivizing High Quality Crowdwork
Chien-Ju Ho, Aleksandrs Slivkins, Siddharth Suri, and Jennifer Wortman Vaughan
In the 24th International World Wide Web Conference (WWW), 2015
Nominee for Best Paper Award
- Adaptive Contract Design for Crowdsourcing Markets:
Bandit Algorithms for Repeated Principal-Agent Problems
Chien-Ju Ho, Aleksandrs Slivkins, and Jennifer Wortman Vaughan
In the 15th ACM Conference on Economics and Computation (EC), 2014 - Adaptive Task Assignment for Crowdsourced Classification
Chien-Ju Ho, Shahin Jabbari, and Jennifer Wortman Vaughan
In the 30th International Conference on Machine Learning (ICML), 2013 - Online Task Assignment in Crowdsourcing Markets
Chien-Ju Ho and Jennifer Wortman Vaughan
In the 26th Conference on Artificial Intelligence (AAAI), 2012 - Towards Social Norm Design for Crowdsourcing Markets
Chien-Ju Ho, Yu Zhang, Jennifer Wortman Vaughan, and Mihaela van der Schaar
In the 4th Human Computation Workshop (HCOMP), 2012 - DevilTyper: A Game for CAPTCHA Usability Evaluation
Chien-Ju Ho, Chen-Chi Wu, Kuan-Ta Chen, Chin-Laung Lei
In ACM Computers in Entertainment, 2011 - KissKissBan: A Competitive Human Computation Game for Image Annotation (Short Paper)
Chien-Ju Ho, Tao-Hsuan Chang, Jong-Chuan Lee, Jane Yung-jen Hsu, Kuan-Ta Chen
In the ACM SIGKDD Workshop on Human Computation (HCOMP), 2009
Also appeared in ACM SIGKDD Explorations Newsletter: Volume 12 Issue 1, June 2010 - On Formal Models for Social Verification
Chien-Ju Ho, Kuan-Ta Chen
In the ACM SIGKDD Workshop on Human Computation (HCOMP), 2009 - PhotoSlap: A Multi-player Online Game for Semantic Annotation
Chien-Ju Ho, Tsung-Hsiang Chang, Jane Yung-jen Hsu
In the 22nd Conference on Artificial Intelligence (AAAI), 2007 - The PhotoSlap Game: Play to Annotate (Intelligent System Demo)
Tsung-Hsiang Chang, Chien-Ju Ho, Jane Yung-jen Hsu
In the 22nd Conference on Artificial Intelligence (AAAI), 2007
- CSE 417T - Introduction to Machine Learning
- Fall 2024, Fall 2022, Spring 2022, Spring 2021, Spring 2020, Fall 2018, Fall 2017
- CSE 518A - Human-in-the-Loop Computation / Crowdsourcing and Human Computation
- Spring 2024, Fall 2022, Fall 2021, Fall 2020, Fall 2019, Spring 2019
Conference Services
- Doctoral Consortium Co-Chair: HCOMP 2022
- Works-in-Progress & Demonstration Co-Chair: HCOMP 2019
- Area Chair or Senior Program Committee: ICLR 2025, AAAI 2025, NeurIPS 2024, ICML 2024, ICLR 2024, AAAI 2024, NeurIPS 2023, ICML 2023, AAAI 2023, NeurIPS 2022, AAAI 2022, NeurIPS 2021, IJCAI 2021, AAAI 2021, AAAI 2020
- Program Committee or Formal Reviewer: HCOMP 2024, AIES 2023, FAccT 2023, TheWebConf 2023, ICML 2022, IJCAI 2022, TheWebConf 2022, ICML 2021, TheWebConf 2021, WSDM 2021, NeurIPS 2020, IJCAI 2020, NeurIPS 2019, TheWebConf 2019, AAAI 2019, NeurIPS 2018, HCOMP 2018, EC 2018, IJCAI 2018, AAAI 2018, TheWebConf 2018, NIPS 2017, EC 2017, WWW 2017, NIPS 2016, HCOMP 2016, EC 2016, IJCAI 2016, NIPS 2015, EC 2015, IJCAI 2015, NIPS 2014, NIPS 2013, AAAI 2013, CrowdRec 2013
- Auxiliary Reviewer or Subreviewer: FAMAS 2019, WINE 2016, AAAI 2014, WINE 2013, HCOMP 2013 WiP, HCOMP 2012
-
HCOMP Workshop on Mathematical Foundations of Human Computation, 2016
with Shuchi Chawla, Michael Kearns, Santosh Vempala, and Jenn Wortman Vaughan -
NIPS Workshop: Crowdsourcing and Machine Learning, 2014
with Adish Singla, David Parkes, Nihar Shah, and Dengyong Zhou
- ACM Transactions on Intelligent Systems and Technology
- ACM Transactions on Economics and Computation
- Annals of Mathematics and Artificial Intelligence
- Artificial Intelligence Journal
- IEEE Transactions on Computational Intelligence and AI in Games
- IEEE Transactions on Computational Social Systems
- IEEE Transactions on Knowledge and Data Engineering
- IEEE Transactions on Parallel and Distributed Systems
- Journal of Autonomous Agents and Multi-Agent Systems
- Journal of Artificial Intelligence Research
- Journal of Machine Learning Research
- Journal of the Association for Information Science and Technology
- World Wide Web Journal
- UCLA Social Computing Reading Group, Sep. 2011 to Mar. 2012