Spaces:
Runtime error
Runtime error
| <html> | |
| <head> | |
| <link rel="preconnect" href="https://fonts.googleapis.com" /> | |
| <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin /> | |
| <link href="https://fonts.googleapis.com/css2?family=Source+Sans+Pro:wght@400;600;700&display=swap" rel="stylesheet" /> | |
| <title>Visual Question Answering (VQA) for Medical Imaging</title> | |
| <style> | |
| * { | |
| box-sizing: border-box; | |
| } | |
| body { | |
| font-family: 'Source Sans Pro', sans-serif; | |
| font-size: 16px; | |
| } | |
| .container { | |
| width: 100%; | |
| margin: 0 auto; | |
| } | |
| .title { | |
| font-size: 24px ; | |
| font-weight: 600 ; | |
| letter-spacing: 0em; | |
| text-align: center; | |
| color: #374159 ; | |
| } | |
| .subtitle { | |
| font-size: 24px ; | |
| font-style: italic; | |
| font-weight: 400 ; | |
| letter-spacing: 0em; | |
| text-align: center; | |
| color: #1d652a ; | |
| padding-bottom: 0.5em; | |
| } | |
| .overview-heading { | |
| font-size: 24px ; | |
| font-weight: 600 ; | |
| letter-spacing: 0em; | |
| text-align: left; | |
| } | |
| .overview-content { | |
| font-size: 14px ; | |
| font-weight: 400 ; | |
| line-height: 33px ; | |
| letter-spacing: 0em; | |
| text-align: left; | |
| } | |
| .content-image { | |
| width: 100% ; | |
| height: auto ; | |
| } | |
| .vl { | |
| border-left: 5px solid #1d652a; | |
| padding-left: 20px; | |
| color: #1d652a ; | |
| } | |
| .grid-container { | |
| display: grid; | |
| grid-template-columns: 1fr 2fr; | |
| gap: 20px; | |
| align-items: flex-start; | |
| margin-bottom: 1em; | |
| } | |
| @media screen and (max-width: 768px) { | |
| .container { | |
| width: 90%; | |
| } | |
| .grid-container { | |
| display: block; | |
| } | |
| .overview-heading { | |
| font-size: 18px ; | |
| } | |
| } | |
| </style> | |
| </head> | |
| <body> | |
| <div class="container"> | |
| <h1 class="title">Visual Question Answering (VQA) for Medical Imaging</h1> | |
| <h2 class="subtitle">Kalbe Digital Lab</h2> | |
| <section class="overview"> | |
| <div class="grid-container"> | |
| <h3 class="overview-heading"><span class="vl">Overview</span></h3> | |
| <div> | |
| <p class="overview-content"> | |
| This project addresses the challenge of accurate and efficient medical imaging analysis in healthcare, | |
| aiming to reduce human error and workload for radiologists. The proposed solution involves developing advanced AI | |
| models for Visual Question Answering (VQA) to assist healthcare professionals in analyzing | |
| medical images (radiology images) quickly and accurately. We fine-tune HuggingFace multimodal model Idefics2-8b using radiology VQA datasets. | |
| </p> | |
| </div> | |
| </div> | |
| <div class="grid-container"> | |
| <h3 class="overview-heading"><span class="vl">Dataset</span></h3> | |
| <div> | |
| <p class="overview-content"> | |
| We fine-tune pre-trained model using these datasets : | |
| </p> | |
| <ul> | |
| <li><a href="https://huggingface.co/datasets/flaviagiammarino/vqa-rad" target="_blank">VQA-RAD dataset</a></li> | |
| <li><a href="https://huggingface.co/datasets/mdwiratathya/SLAKE-vqa-english" target="_blank">SLAKE dataset</a></li> | |
| <li><a href="https://huggingface.co/datasets/mdwiratathya/ROCO-radiology" target="_blank">ROCO dataset</a></li> | |
| </ul> | |
| </div> | |
| </div> | |
| <div class="grid-container"> | |
| <h3 class="overview-heading"><span class="vl">Model Architecture</span></h3> | |
| <div> | |
| <p class="overview-content">The model is trained using Idefics2-8b.</p> | |
| <img class="content-image" src="https://raw.githubusercontent.com/Kalbe-x-Bangkit/C24-RM-Kalbe-Bangkit/main/img/idefics2_architecture.png" alt="model-architecture" /> | |
| </div> | |
| </div> | |
| </section> | |
| <h3 class="overview-heading"><span class="vl">Demo</span></h3> | |
| <p class="overview-content">Please upload an image and question or select from the examples to see the answer prediction</p> | |
| </div> | |
| </body> | |
| </html> | |