Workshop on Interactive Learning with Implicit Human Feedback

About
Systems that can learn interactively from their end-users are quickly becoming widespread in real-world applications. Typically humans provide tagged rewards or scalar feedback for such interactive learning systems. However, humans offer a wealth of implicit information (such as multimodal cues in the form of natural language, speech, eye movements, facial expressions, gestures etc.) which interactive learning algorithms can leverage during the process of human-machine interaction to create a grounding for human intent, and thereby better assist end-users. A closed-loop sequential decision-making domain offers unique challenges when learning from humans -– (1) the data distribution may be influenced by the choices of the algorithm itself, and thus interactive ML algorithms need to adaptively learn from human feedback, (2) the nature of the environment itself changes rapidly, (3) humans may express their intent in various forms of feedback amenable to naturalistic real-world settings, going beyond tagged rewards or demonstrations. By organizing this workshop, we attempt to bring together interdisciplinary experts in interactive machine learning, reinforcement learning, human-computer interaction, cognitive science, and robotics to explore and foster discussions on such challenges. We envision that this exchange of ideas within and across disciplines can build new bridges, address some of the most valuable challenges in interactive learning with implicit human feedback, and also provide guidance to young researchers interested in growing their careers in this space.

Some potential questions we hope to discuss at this workshop are listed below:

When is it possible to go beyond reinforcement learning (with hand-crafted rewards) and leverage interaction-grounded learning from arbitrary feedback signals where grounding for such feedback could be initially unknown, contextual, rich and high-dimensional?
How can we learn from natural/implicit human feedback signals such as natural language, speech, eye movements, facial expressions, gestures etc. during interaction? Is it possible to learn from human guidance signals whose meanings are initially unknown or ambiguous? Even when there is no explicit external reward?
How should learning algorithms account for a human’s preferences or internal reward that is non-stationary and changes over time? How can we account for non-stationarity of the environment itself?
How much of the learning should be pre-training (i.e. learning for the average user) versus how much should it be interactive or personalized (i.e. for finetuning to a specific user)?
How can we develop a better understanding of how humans interact with/ teach other humans or machines? And how could such an understanding lead to better designs for learning systems that leverage human signals during interaction?
How to design intrinsic reward systems that could push agents to (learn to) become socially integrated/coordinated/aligned with humans?
How can well-known design methods from HCI (such as ability-based design) be imported and massively used in AI/ML? What is missing from today’s technological solution paradigms that can allow for ability-based design to be deployed at scale? How can the machine learning community assist HCI and accessibility research communities to build adaptive learning interfaces targeting a wide range of marginalized and specially-abled sections of society?
What are the minimal set of assumptions under which learning from arbitrary/implicit feedback signals is possible for the interaction-grounded learning paradigm?

Time (GMT-10)
09:00 am - 09:10 am		Organizers Introductory Remarks
09:10 am - 10:30 am		Invited Speakers 1-4 TBD
10:30 am - 11:00 am		Contributed Talks
11:00 am - 01:00 pm		Lunch/virtual social event
01:00 pm - 02:30 pm		Invited Speakers 5-8 TBD
02:30 pm - 03:00 pm		Contributed Talks
03:00 pm - 04:00 pm		Location: TBD Poster Session
04:00 pm - 05:20 pm		Panel Session
05:20 pm - 05:30 pm		Organizers Concluding Remarks

Call for papers
New: The call for papers is now open.

Areas of interest

We solicit submissions related to (but not limited to) the following themes on interaction-grounded machine learning with humans:

Leveraging different types of human input modalities for interactive learning
Models and representations learned from human data
Online learning algorithms for human-machine collaboration
Personalized interaction-based learning
Theoretical advances for interactive learning with implicit human feedback
Interactive Learning with non-stationary rewards and environment dynamics
Applications for HCI and accessibility
Understanding how humans teach other humans and learning agents/embodied robots

All submissions will be managed through OpenReview. The review process is double-blind so the submission should be anonymized. Papers should be a maximum of 8 pages (excluding references), and formatted in ICML style. Accepted papers will be presented as posters during the workshop and select works will be invited to give spotlight talks during the workshop. Accepted papers will be made available online on the workshop website as non-archival reports, allowing submissions to future conferences or journals.

Authors may optionally add appendices in their submitted paper. Supplementary Materials uploads are to only be used optionally for extra videos/code/data/figures and should be uploaded separately in the submission website.

Submissions will be evaluated based on novelty, rigor, and relevance to theme of the workshop. Both empirical and theoretical contributions are welcome. All participants must adhere to the ICML Code of Conduct.

Important Dates

Submission deadline: May ~~24th~~ 31st, 2023, AoE.
Author Notifications: June 18th, 2023, AoE.
Camera Ready: July 10th, 2023, AoE.
Workshop: July 29th, 2023.