Multimodal data is available in many applications like e-commerce production listings, social media posts and short videos. However, existing algorithms dealing with those types of data still focus on uni-modal representation learning by vision-language alignment and cross-modal retrieval. In this workshop, we target to bring a new retrieval problem where both queries and documents are multimodal. With the popularity of vision language modeling,  large language models (LLMs), retrieval augmented generation (RAG), and multimodal LLM, we see a lot of new opportunities for multimodal representation and retrieval tasks. This event will be a comprehensive half-day workshop focusing on the subject of multimodal representation and retrieval. The agenda includes keynote speeches, oral presentations, and an interactive panel discussion.

Our objective with this workshop is to capture the interest of researchers in this emerging challenge. We anticipate that the workshop will serve as a catalyst for establishing a dedicated community focused on this topic. By highlighting the novelty and significance of the problem, we aim to attract researchers who are eager to explore and contribute to this field. We invite original research & industrial application papers that present research on multimodal data representation and retrieval.

Submission Guidelines

Submissions of short papers must be in English, in PDF format, and be at most 4 pages (including figures, tables, proofs, appendixes, acknowledgments, and any content except references) in length, with unrestricted space for references, in the current ACM two-column conference format. Suitable LaTeX, Word, and Overleaf templates are available from the ACM Website (use “sigconf” proceedings template for LaTeX and the Interim Template for Word). ACM’s CCS concepts and keywords are required for review.

For LaTeX, the following should be used:

    \documentclass[sigconf,natbib=true,anonymous=true]{acmart}

Submissions must be anonymous and should be submitted electronically via EasyChair:

Important dates for submissions to MRR 2024

- Workshop paper submission due date: April 25, 2024 (11:59 pm, AOE), May 5, 2024 (11:59 pm, AOE)

- Workshop paper acceptance notification: May 23, 2024

- Workshop day: July 18, 2024

Topics includes but not limited to

• Multimodal representation learning and retrieval, such as
  • Multimodal embeddings learning and fusion 
  • Learning with noisy labels 
  • Multimodal query representation 
  • Multimodal query understanding 
  • Multimodal query suggestion 
  • Ranking algorithms for multimodal retrieval 
• Dataset, such as 
  • New dataset for multimodal retrieval 
  • Ways to synthesize data 
• Applications of Multimodal Retrieval, such as 
  • Multimodal retrieval in search engine 
  • Multimodal retrieval in recommendation system 
  • Multimodal retrieval in Ads 
  • Multimodal retrieval in Chatbot
  • Multimodal retrieval in Robotics

Organizers

• Xinliang Zhu, Amazon
• Arnab Dhua, Amazon
• Douglas Gray, Amazon
• I. Zeki Yalniz, Meta
• Tan Yu, Nvidia
• Mohamed H. Elhoseiny, KAUST
• Bryan A. Plummer, Boston University

Venue

The conference will be held on Thursday, July 18, 2024, in conjunction with SIGIR 2024, in Washington D.C..

Contact

All questions about submissions should be emailed to: xlzhu@amazon.com.