Developing an interactive, collaborative mixed reality(MR) experience typically requires professional skills, even at the prototype stage. In this paper, we present DreamPortal, a mixed reality application integrated with generative AI pipelines that convert speech into virtual objects and support multiuser interaction. We also integrate a Large Language Model(LLM) to automate reasoning about the interaction functions within generated objects and physical scenes.
ACME Lab
