Baijun Xie

Baijun Xie

Currently, I am a research engineer at Meta Reality Labs. My mission is to research new augmented reality (AR) input devices with a focus on human-computer interaction (HCI), input sensing, machine learning, and user interfaces. I am working with a diverse and highly interdisciplinary team of researchers and engineers and will have access to cutting-edge technology, resources, and testing facilities.

Previsouly, I am a graduate research assistant at the George Washington University. I was conferred with a Doctor of Philosophy degree by the end of the year 2023. My doctoral research is at the intersection of machine learning and human-robot interaction (HRI). From a methodological perspective, the objective of my doctoral study is to utilize machine learning algorithms together with emotional intelligence to facilitate the creation of more engaging and empathetic experiences in HRI. Throughout the duration of my doctoral research, the robotic system that I have developed has successfully attained the capability of engaging in interactive emotional communication by utilizing both verbal and non-verbal social cues. Additionally, I had the task of designing and assessing a social-emotional robot that incorporated an interactive gaming scenario. In addition, a user study was conducted on the empathetic robotic companion that has been designed specifically for autistic adolescents, along with a personalized intervention program.

🔥 News

  • 2024.04:  💻 💻 I am thrilled to start a new position as a research engineer at Meta Reality Labs!
  • 2023.11:  🎉🎉 I have pased my PhD dissertation defense!

💻 Experience

  • 2024.04 - Present, Research Engineer, Meta Reality Labs
  • 2020.06 - 2023.12, Graduate Research Assistant, The George Washington University
  • 2018.09 - 2020.05, Graduate Teaching Assistant, The George Washington University
    • SEAS 1001 Engineering Orientation
    • BME 2825 Biomedical Engineering Programming II
  • 2019.01, Mentor, 2019 George Hacks Medical Solutions Hackathon
  • 2017.02 - 2018.05, Graduate Researcher, The George Washington University
  • 2014.10 - 2016.05, Undergraduate Researcher, Shenzhen University
    • Design of Perovskite Solar Cells
    • Laboratory Open Fund Project of Shenzhen University
    • Shenzhen University 2015 Challenge Cup University Student Venture Contest
  • 2015.07 - 2015.08, Intern, Shenzhen JBT Smart Lighting Co., Ltd.,

📝 Projects and Publications

Social Intelligence

ICCV 2023
sym

Multi-Modal Correlated Network with Emotional Reasoning Knowledge for Social Intelligence Question-Answering

Baijun Xie, Chung Hyuk Park, Proceedings of the IEEE/CVF International Conference on Computer Vision [Code]

  • We participated in the Social IQ 2.0 Challenge, which is designed to benchmark recent AI technologies’ skills to reason about social interactions, which is referred to as Artificial Social Intelligence.
  • We developed a framework named Multi-Modal Temporal Correlated Network with Emotional Social Cues (MMTC-ESC). MMTC-ESC exhibits an attention-based mechanism to model cross-modal correlations and utilizes contrastive learning for reasoning about emotional social cues.

Human-Robot Interaction

CAREER
sym

2019.03 - 2023.12:   As a core member, I participated in the project of CAREER: Social Intelligence with Contextual Ambidexterity for Long-Term Human-Robot Interaction and Intervention (LT-HRI2).

  • The goal of this project was to understand the fundamental principles of human interactions and behaviors and translate these mechanisms into computational modeling and algorithms for a novel assistive robotic framework. Toward this goal, we developed a socially assistive robotic framework with contextual ambidexterity that is perceptive of personal socio-emotional states, capable of learning social skills, emotionally interactive, and gender-smart for long-term human-robot interaction and intervention. Several publications related to this project include:

Emotion Recognition

EMOTION
sym

To equip the robotic agent with emotional intelligence, we conducted comprehensive studies on multimodal emotion recognition, integrating facial expressions, speech text, and voice. Our approach involved training deep learning models to fuse these diverse inputs, enabling the agent to accurately interpret human emotions. This research aimed to enhance HRI by enabling context-aware, empathetic responses, ultimately improving user engagement and trust in robotic systems.

  • We developed a cutting-age multimodal framework for emotion recognition by fine-tuning backbone models and integrating a cross-modality fusion module to enhance feature integration and predictive accuracy.
  • We introduced a sinusoidal model-based spectral feature extraction method that enhances arousal and valence prediction in music emotion recognition, reducing prediction errors in CNN and regression models.

Medical Imaging

NERVE
sym
animated gif

2020.09 - 2022.12:   As a core member, I participated in the project of Real-Time Nerve Tissue Detection and Visualization.

  • We proposed the use of birefringence images with deep learning inference to help with the detection of nerves. The study findings showed that leveraging birefringence images outperforms its RGB counterpart for nerve detection and segmentation. Additionally, introducing the cross-attention module on a multi-modality network improved the detection and segmentation of nerve structures on the birefringence images.

Robotics

ROBOT
sym
ROBOT
animated gif

2017.02 - 2018.05:   I participated in the project of “Multi-Domain Search and Rescue using Cooperative Robots” during my Master’s research. In this project, we

  • Developed a collaborative system integrating aerial quadrotors and ground robots for efficient rescue operations, utilizing advanced computer vision for autonomous detection and tracking and ensuring seamless inter-platform communication.
  • Applied a model-based least squares pose estimation method and trained a custom CNN for precise recognition of ground robots, enhancing detection and tracking even in challenging conditions.
  • Conducted rigorous experiments and simulations on quadrotor control, employing both PID and geometry controllers to achieve autonomous flight stability and precise navigation.

Numerical Analysis

SWE
swe

2017.08 - 2017.12:   I have done an independent project that deriving the full vectorization of a system of partial differential equations, Shallow Water Equations (SWEs), with conservative form. [Code]

  • Simulated the numerical results of SWEs and analyzed the grid convergence with different numerical schemes; improved the order of convergence from 1 to 2 by utilizing a Lax-Wendroff scheme.

Others

🎖 Honors and Awards

  • 2023.01 2022 Collins Distinguished Doctoral Fellowship at the GWU.
  • 2022.04 2022 GW Technology Commercialization Innovation Competition (Audience’s choice posters Prize).
  • 2022.01 2021 Collins Distinguished Doctoral Fellowship at the GWU.
  • 2016.05 2016 A Hundred Excellent Final Year Theses Prize in Shenzhen University .
  • 2015.09 2015 Shenzhen University Challenge Cup University Student Venture Contest (2nd Prize).
  • 2014.09 Laboratory Open Fund Project of Shenzhen University (3rd Prize).

📖 Educations

  • 2018.09 - 2023.12, Doctor of Philosophy, The George Washington University, Washington DC, U.S.A.
    • Related courses: Probability for Computer Science, Design & Analysis of Algorithm, Machine Learning, Pattern Recognition, Digital Image Processing, Robotics Vision and Perception, Biomedical Signal Analysis
  • 2016.09 - 2018.05, Master of Science, The George Washington University, Washington DC, U.S.A.
    • Related courese: Analytical Methods in Engineering, Electromechanical Control System, Robotic Systems, Mechatronics Design, Numerical Solution Tech in MAE, Applied Nonlinear Control, Applied Optimal Control & Estimation, Spacecraft Attitude Control, System Optimization, Advanced Mechanical Engineering Design
  • 2012.09 - 2016.06, Bachelor of Engineering, Shenzhen University, Guandong, China.

💬 Invited Talks

Oral

  • 2023.10, 2023 IEEE/CVF International Conference on Computer Vision (ICCV) Workshops at Paris, France
  • 2022.11, 2022 BME Day in the Department of Biomedical Engineering at the GWU.
  • 2021.11, 2021 the Artificial Intelligence for Human-Robot Interaction AAAI Fall Symposium | [video]
  • 2020.04, 2020 ACM/IEEE International Conference on Human-Robot Interaction | [video]

Poster

  • 2023.04, 2023 GW SEAS R&D Showcase at the GWU
  • 2023.03, 2023 ACM/IEEE International Conference on Human-Robot Interaction at Stockholm, SE
  • 2022.05, 2022 GW Technology Commercialization Innovation Competition at the GWU
  • 2021.07, 2021 18th International Conference on Ubiquitous Robots (online)
  • 2019.10, 2019 Biomedical Engineering Society (BMES) Annual Meeting at Philadelphia
  • 2019.10, 2019 Research and Development Showcase at the GWU
  • 2019.04, 2019 Research Day at the GWU
  • 2018.02, 2018 GW SEAS R&D Showcase at the GWU