We aim to provide a complete user experience with integrated medication and doctor-appointment APIs as well as awareness surrounding Skin cancer and allergies. Using 3 state-of-the-art Deep Learning Classification models we try to predict with considerably high accuracy if you are suffering from Melanoma (which is the most fatal type of skin cancer) or any other type of cancer (basal cell carcinoma, and 7 other types), or if you are suffering from an allergy (using a separate model). With features like, embedded videos, remedies, medicine browsing, Find a Doctor we hope to be a one-stop solution for the user's dermatological problems.
We have used 3 models for 3-layered multipurpose testing, namely, for detecting Melanoma(the most fatal type of Skin Cancer), the given types of Skin Cancers:
-
Melanocytic nevi
-
Benign keratosis-like lesions
-
Basal cell carcinoma
-
Actinic keratoses
-
Vascular lesions
-
Dermatofibroma
And also the given types of allergies and skin diseases:
-
Acne and Rosacea
-
Cellulitis, Impetigo, and other Bacterial Infections
-
Eczema and Atopic Dermatitis
-
Nail Fungus and other Nail Disease
-
Application/Deployment:
To deploy our models we have seamlessly integrated Tensorflow models using TFLite framework into a Flutter Android App, where we have used multiple APIs to create an engaging user-experience and provided with testing/educative as well as preventive and medical aid features. The salient features of our Application is listed as follows:
-
Carousel to showcase App features and provide an overview
-
User input form to check symptoms and select model for prediction
-
3-types of cutting edge deep learning prediction model for reliable results
-
Disease information cards to spread awareness and knowledge
-
Contact Doctor through Email and Whatsapp APIs
-
Search for Local Hospitals using Google Map integration
-
Real-time database for appointment scheduling
The three models were trained on open source datasets, some using custom or transfer learning architectures on Kaggle Cloud GPUs. For specific training and model architecture refer to the training notebooks in Models/. The data pre-processing, scraping, augmentation techniques are also described in these notebooks. Models were trained using Tensorflow 2.0 deep learning framework, and were exported in .tflite (Tensorflow lite) format. For post-processing we have tested the models, then applied quantisation, pruning to clip inconsequential weights in the model an reduce it's size for deployment. The pre and post quantisation test results can be found in the model training notebooks as well.
-
Model Training, Data Preprocessing and Conversion to TFLite is shown in Skin-Cancer-Detection-App/Models/.. as Jupyter Notebooks. Data is taken from Kaggle for Melanoma Model and Cancer Model. For Allergy Model a custom scraper is implemented for obtaining data, which is shown in, Skin-Cancer-Detection-App/Models/allergy-model/..
-
Flutter app source code is contained in Skin-Cancer-Detection-App/App/..
-
Models in .tflite formats and their labels are in App/assets/
To clone and use this repository, you'll need Git Large File Storage (LFS).
Our Developer Guide explains how to set up Git LFS for LSST development.
How to run our App? Just download the apk and install it. Alternatively you can run the source code in Android Studio, by connecting an AVD and using your own google services login. To run the app in the emulator go to app directory and execute: flutter run. Flutter and Dart SDK required.
Solution was partly inspired by SIIM-ISIC Melanoma Detection Challenge on Kaggle, in which we had won rank 101 of 3314 teams and silver medal. The app was submitted to Hacker-Earth's AGBI Health-tech grand challenge, organised by Virtusa technologies, Niti Aayog, and Mehta group of hospitals. Submission was awarded 2nd prize out of 100 shortlisted ideas and 1200+ registered teams, and an incubation offer.