skip to main content

Find a Project

Back to Project Search

Conversation 3D Visualization on the Web

Project

Project Details

Program
Computer Science
Field of Study
visual computing
Division
Computer, Electrical and Mathematical Sciences and Engineering
Faculty Lab Link
https://nanovis.kaust.edu.sa/

Project Description

Conversational visualization can be characterized as an integration of interactive visualization with large-language models (LLM) through prompting or fine-tuning strategies. We have recently developed several web-based visualization systems for volume visualization (tomography.kaust.edu.sa) and for visualization of atomistic models (mesocraft.kaust.edu.sa). The task for this project will be to integrate these visualization systems with client-based web-based LLMs (or alternatively with server-based LLM inference) and with speech recognition, so that these conversational capabilities extend the interaction metaphors based on mouse and keyboard interactions.

About the Researcher

Ivan Viola
Professor, Computer Science
Computer, Electrical and Mathematical Science and Engineering Division

Affiliations

Education Profile

  • Senior Lecturer of Computer Science (venia docendi), TU Wien, Austria (2016)
  • PhD in Computer Science (Dr. techn.), TU Wien, Austria (2005)
  • MSc in Computer Science (Dipl.-Ing.), TU Wien, Austria (2002)

Research Interests

The research interest of Professor Viola is scalable technology for interactive molecular visualization with the ultimate goal of constructing, visualizing, and modelingA  the entire complex biological cell at atomistic detail. This technology will allow people to interact, explore, study, and understand the life at nanoscale.

Selected Publications

  • D. KouA™il, L. AŒmolA­k, B. KozlA­kovA¡, H.Y. Wu, G. Johnson, D. Goodsell, A. Olson, M. E. GrA¶ller, I. Viola. Labels on Levels: Labeling of Multi-Scale Multi-Instance and Crowded 3D Biological Environments, IEEE Transactions on Visualization and Computer Graphics, 2019
  • H. Miao, E. De Llano, T. Isenberg, E. GrA¶ller, I. Barisic, I. Viola. DimSUM: Dimension and Scale Unifying Maps for Visual Abstraction of DNA Origami Structures, Computer Graphics Forum (Proceedings of Eurovis), 37(3), 2018
  • H. Miao, E. De Llano, J. Sorger, Y. Ahmadi, T. Kekic, T. Isenberg, M. E. GrA¶ller, I. Barisic, I. Viola. Multiscale Visualization and Scale-adaptive Modification of DNA Nanostructures, IEEE Transactions on Visualization and Computer Graphics, 24(1), 2018
  • I. Viola, T. Isenberg. Pondering the Concept of Abstraction in (Illustrative) Visualization, IEEE Transactions on Visualization and Computer Graphics, (99), 2017
  • T. Klein, L. Autin, B. Kozlikova, D. Goodsell, A. Olson, M. E. GrA¶ller, I. Viola. Instant Construction and Visualization of Crowded Biological Environments. IEEE Transactions on Visualization and Computer Graphics, 24(1), 2017.

Desired Project Deliverables

The outcome of the project is a web-based conversational system, capable of turning user's verbal queries into 3D interactions that control the visualization mapping parameters.

Recommended Student Background

3D computer graphics background from computer science perspective
C++ and GPU shader coding skills
Experiences with LLM client or server-based deployment
Calculus, multivariate calculus, linear algebra, probability and statistics
On this page