Dermadex App: photo-based AI dermatologic diagnosis

Why artificial intelligence? Firstly, it's the only way to provide a diagnosis based on a photograph of a problematic skin area. Neural networks enable automating the process and offering users basic recommendations. Secondly, utilizing AI as a telemedicine tool is a powerful PR move for promoting the application.

At the outset, Rami and Dr. Khoddami defined the main user scenario:

1. The user with a problematic skin area takes a photo of it and uploads it to the application.

2. Artificial intelligence provides a preliminary diagnosis, identifies the condition, and recommends treatment options.

3. Immediately afterward, the user can schedule a consultation with a professional dermatologist.

The application is free: patients receive recommendations from artificial intelligence. The primary monetization model involves dermatologists who register on the platform and confirm their qualifications paying a small commission for each paid consultation. Additionally, Rami has planned other monetization options: standard in-app advertising and sponsored articles in the "Helpful Materials" section.

The task is ambitious — to turn the neural network into a professional diagnostician for skin diseases. With this in mind, Rami turned to Upwork, and here comes Unistory into play!

Our managers have worked out the project structure and selected technologies: for the web version, we decided to use React, for the mobile application — React Native, and to implement the overall backend in C#.

Together with the client, we immediately agreed that our primary focus would be on finding datasets, selecting the appropriate AI model, and training it. Everything else comes later. First, we need a killer feature!

The first stage of development was the preparation of the Proof of Concept (PoC). We often deal with experimental projects, more frequently involving blockchain and AI technologies, which is why we have a dedicated R&D engineer for creating PoCs. It is this individual who tests the riskiest technical hypotheses, thereby saving money and time on the development of the entire product.

At this stage, we decided that to test hypotheses and develop the Proof of Concept, it would be sufficient to train the neural network to identify six groups of diseases. The next task is to find data (datasets) for training the neural networks.

We explored open-source repositories and found 21 datasets. To our disappointment, there wasn't as much high-quality material in them as we had hoped. To address this issue, we decided to rely on zero-shot and few-shot testing.

Developers tested several AI models and chose CLIP — a base model capable of classifying images, identifying objects within them, and generating text based on the images.

1. Zero-shot: Evaluated the capabilities of models without prior training on datasets. This means that the model was tested on tasks or datasets it had not encountered before.

2. Fine-tuning (few-shot): In this approach, models underwent additional training on our datasets.

The trained neural network achieved a 99% accuracy in diagnosis given high-quality photographs! We taught CLIP to identify over 60 different diseases grouped into six major categories: acne, psoriasis, rosacea, eczema, herpes, and vitiligo.

The main challenge faced by the developers remained consistent — a shortage of data and insufficiently high-quality datasets available in the public domain.

The solution came in the form of data augmentation — we expanded the database by generating artificial data based on real ones. By introducing minor distortions into the images, we were able to significantly enhance the dataset for model training. We are currently continuing our search for datasets for training, with the goal of teaching CLIP to work with 30 disease groups.

While our R&D engineer was working on the Proof of Concept, and managers were preparing the structure, the client was dealing with legal matters. Rami studied what the application should be like to pass HIPAA standardization for future launch in the USA. HIPAA certification indicates that the product meets security requirements regarding patient data.

To obtain HIPAA certification, we preplanned logging everything: every user request must be recorded in the database. Information must be stored about all actions of patients, doctors, and administrators. However, nobody should have access to patient data.

Rami is currently preparing to launch the application in Canada. After that, we plan to scale into the US and European markets. The launch will proceed gradually, as each country has its own heathcare nuances.