Hello, I'm Sumin Shim! 👋
I am an M.S. student in AIE, ECE at Carnegie Mellon University. My research interests center on Multimodal AI, Generative Video Model Control and World Models. I focus on bridging high-level LLM/VLM reasoning with physical system control and scene prediction to build safe, intelligent autonomous agents.
Education
-
Carnegie Mellon University
-
Yonsei University
Publications
-
Anchoring and Rescaling Attention for Semantically Coherent Inbetweening CVPR 2026 Highlight
-
Do MLLMs Capture How Interfaces Guide User Behavior? A Benchmark for Multimodal UI/UX Design Understanding ACL 2026
-
Towards Visual Text Design Transfer Across Languages NeurIPS 2024
* Equal contribution
Experience
-
OptAI
- Finetuned a ~1B-parameter Korean language model for the LG U+ call summarization project.
- Deployed the finetuned model on-device and built the company's demo mobile app.
- Demonstrated the on-device LLM demo to external partners at the 2025 K-ICT exhibition.
-
Multimodal AI Laboratory, Yonsei University
- Conducted research on Vision-Language Models (VLMs) and Large Language Models (LLMs).
- Collaborated on projects including visual text design transfer and multimodal UI/UX understanding.
-
Prometheus AI Student Club
- Led an intercollegiate AI club, coordinating project teams building AI apps and research prototypes.
- Organized a 200-participant AI hackathon (KRW 4M prize pool) and two demo days with industry sponsors.
-
HuemoneLab
- Taught Python, Java, and basic algorithms to beginner and intermediate students.
Projects
-
AkaLlama: Educational Korean LLM for Yonsei Students
Built a large-scale Korean LLM dataset and deployed an ELO ranking-based pairwise evaluation system.
-
Know Yourself: Building an LLM with Strong Knowledge of AI
Developed an AI-knowledge QA benchmark and fine-tuned a small language model on a domain-specific corpus.
-
WorkMate: AI Legal Chatbot for Foreign Workers
Developed the database and Retrieval-Augmented Generation (RAG) system for a legal chatbot.
-
Detective Game with LLM Persona Agents
Interactive detective game utilizing custom LLM persona agents.
Awards & Scholarships
Honors & Awards
- 2nd Place AI Challenge Season 2: Innovating the World with AI (2025)
- 3rd Place AI Convergence Policy Hackathon (2025)
- 3rd Place 3rd KRX Financial Language Model Performance Competition (2024)
- 4th Place DOB Deep Learning Challenge (Face Swapping Task) (2023)
- 3rd Place 4th Korea Red Cross Sharing & Service Hackathon (2023)
Scholarships
- • Internship Excellence Scholarship, Yonsei University (Oct 2025)
- • University Innovation Scholarship, Yonsei University (Oct 2025; Apr 2023)
- • SW/AI Activity Excellence Scholarship, Yonsei University (Feb 2025)
- • Industry-Academia Project Excellence Scholarship, Yonsei University (Jul 2024)