Hello, I'm Sumin Shim! 👋

I am an M.S. student in AIE, ECE at Carnegie Mellon University. My research interests center on Multimodal AI, Generative Video Model Control and World Models. I focus on bridging high-level LLM/VLM reasoning with physical system control and scene prediction to build safe, intelligent autonomous agents.

Education

  • CMU LogoCarnegie Mellon University

  • Yonsei LogoYonsei University

Publications

  • Inbetweening paper thumbnail

    Anchoring and Rescaling Attention for Semantically Coherent Inbetweening CVPR 2026 Highlight

  • UI UX benchmark paper thumbnail

    Do MLLMs Capture How Interfaces Guide User Behavior? A Benchmark for Multimodal UI/UX Design Understanding ACL 2026

  • Visual text design transfer paper thumbnail

    Towards Visual Text Design Transfer Across Languages NeurIPS 2024

* Equal contribution

Experience

  • OptAI

    • Finetuned a ~1B-parameter Korean language model for the LG U+ call summarization project.
    • Deployed the finetuned model on-device and built the company's demo mobile app.
    • Demonstrated the on-device LLM demo to external partners at the 2025 K-ICT exhibition.
  • Multimodal AI Laboratory, Yonsei University

    • Conducted research on Vision-Language Models (VLMs) and Large Language Models (LLMs).
    • Collaborated on projects including visual text design transfer and multimodal UI/UX understanding.
  • Prometheus AI Student Club

    • Led an intercollegiate AI club, coordinating project teams building AI apps and research prototypes.
    • Organized a 200-participant AI hackathon (KRW 4M prize pool) and two demo days with industry sponsors.
  • HuemoneLab

    • Taught Python, Java, and basic algorithms to beginner and intermediate students.

Projects

  • AkaLlama: Educational Korean LLM for Yonsei Students

    Built a large-scale Korean LLM dataset and deployed an ELO ranking-based pairwise evaluation system.

  • Know Yourself: Building an LLM with Strong Knowledge of AI

    Developed an AI-knowledge QA benchmark and fine-tuned a small language model on a domain-specific corpus.

  • WorkMate: AI Legal Chatbot for Foreign Workers

    Developed the database and Retrieval-Augmented Generation (RAG) system for a legal chatbot.

  • Detective Game with LLM Persona Agents

    Interactive detective game utilizing custom LLM persona agents.

Awards & Scholarships

Honors & Awards

  • 2nd Place AI Challenge Season 2: Innovating the World with AI (2025)
  • 3rd Place AI Convergence Policy Hackathon (2025)
  • 3rd Place 3rd KRX Financial Language Model Performance Competition (2024)
  • 4th Place DOB Deep Learning Challenge (Face Swapping Task) (2023)
  • 3rd Place 4th Korea Red Cross Sharing & Service Hackathon (2023)

Scholarships

  • Internship Excellence Scholarship, Yonsei University (Oct 2025)
  • University Innovation Scholarship, Yonsei University (Oct 2025; Apr 2023)
  • SW/AI Activity Excellence Scholarship, Yonsei University (Feb 2025)
  • Industry-Academia Project Excellence Scholarship, Yonsei University (Jul 2024)