Aye-Eye_AIED (AI in Education) Intervention for Human Bias Metacognitive Design for Critical Thinking

While artificial intelligence (AI) increasingly transforms academic environments, a critical gap exists in understanding how human biases influence AI interactions. This mixed-methods study investigates the metacognitive needs and design requirements for developing an AI literacy tool that promotes critical engagement among university students. Through mixed-method user research, we identified three key user needs: (1) human bias awareness in prompt engineering, (2) understanding of AI capabilities, and (3) support for critical thinking. We examined four dimensions shaping human-AI collaboration—trust, confidence, agency, and anthropomorphism tendency—leading to two distinct user personas. Based on these findings, we developed ‘Aye-Eye,’ an integrated tool combining real-time bias visualisation and prompt refinement capabilities. Our work contributes to human-centred AI research by demonstrating and addressing different user engagement patterns with metacognitive scaffolding to enhance critical thinking, supporting UN Sustainable Development Goals 4 (Quality Education) and 16 (Peace, Justice and Strong Institutions).

This is a project my teammate and I at UCL submitted to CHI2025 Student Design Competition. Try it out: https://cmlmanni.github.io/AyeEye