- Jaivanti Dhokey
- Atharva Alshi
- Sanika Gadge
- Vriddhi Gupta
- Krishna Ashar
- Chanchal Agrawal
- Aishwarya Harkare
- Purva Anjarlekar
The cogneye app opens up with a logo screen, which further moves towards a screen which reads out App features. User can either click on any of the feature or speak out the name of the feature. Eventually the feature will activate camera of the smartphone. Once the Phone's camera is moved around an object/Letter/Paragraph, the voice is enabled and it reads out the object/letter/paragraph aloud. Currently, the app is still in development mode, cogneye team is working efficiently to improve the quality of the app.
Code snippet for splash screen
val SPLASH_TIME_OUT = 4000.toLong()
Executors.newSingleThreadExecutor().execute {
Thread.sleep(SPLASH_TIME_OUT)
startActivity(Intent(this, MainActivity::class.java))
finish()
}Code Snippet for OCR feature.
SparseArray<TextBlock> sparseArray = detections.getDetectedItems();
StringBuilder stringBuilder = new StringBuilder();
for (int i = 0; i<sparseArray.size(); i++){
TextBlock textBlock = sparseArray.valueAt(i);
if (textBlock != null && textBlock.getValue() !=null){
stringBuilder.append(textBlock.getValue() + " ");
}
}
final String stringText = stringBuilder.toString();Code snippet for Object detection
private fun bindPreview(cameraProvider: ProcessCameraProvider){
val preview = Preview.Builder().build()
val cameraSelector =CameraSelector.Builder()
.requireLensFacing(CameraSelector.LENS_FACING_BACK)
.build()
preview.setSurfaceProvider(binding.previewView.surfaceProvider)
val imageAnalysis = ImageAnalysis.Builder()
.setTargetResolution(Size(720,1280))
.setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
.build()
imageAnalysis.setAnalyzer(ContextCompat.getMainExecutor(this),ImageAnalysis.Analyzer{ imageProxy ->
val rotationDegrees = imageProxy.imageInfo.rotationDegrees
val image = imageProxy.image
if (image != null){
val processImage= fromMediaImage(image,rotationDegrees)
objectDetector
.process(processImage)
.addOnSuccessListener { objects ->
for (i in objects){
if(binding.parentLayout.childCount >1) binding.parentLayout.removeViewAt(1)
val element = Draw(context =this,
rect = i.boundingBox,
text = i.labels.firstOrNull()?.text ?: "Undefined")
binding.parentLayout.addView(element,1)
fun processTexttoSpeech() {
val objectname = i.labels.firstOrNull()?.text ?: "Undefined"
tts!!.speak(objectname, TextToSpeech.QUEUE_FLUSH, null, "")
}
processTexttoSpeech()
}
imageProxy.close()
}Code snippet from android manifest file.
<uses-permission android:name="android.permission.CAMERA"></uses-permission>
<uses-permission android:name="android.permission.RECORD_AUDIO"></uses-permission>- GitHub repo link: CognEye Repo
- Drive link: CognEye drive link
- Website link: CognEye Website
Tools and technologies that you learnt and used in the project.
- Java
- Kotlin
- HTML/CSS
- Javascript
- Python
- Android Studio
- CognEye will act as bridge for visually impaired people to see the world through the features like object detection, optical character recognition, facial recognition, etc.
- CognEye will activate camera and voice technologies in smartphones, to identify objects, letters, paragraph, etc around the user.
- It will also provides a inbuilt voice assitance to help the user with understanding the app features and its use.
- Jaivanti Dhokey: "Working with cogneye team was an amazing learning experience for me. Right from brainstorming various ideas and implementing them to deploying ML Models and entire website, Learning more about web development and ML field, has added a lot of wisdom to explore more deeply in this field."
- Atharva Alshi: "Working with the team on this project was a wonderful experience.I along with my teammate Sanika implemented ML model Object Detection using ML-kit.Using TextToSpeech intent we implemented it on our model so the app speaks which object is has detected.Overall experience was really amazing and got to learn alot from the other group members as well. "
- Sanika Gadge: "Overall this was a great experience. I along with my team-mate Atharva specifically worked on Object Detection feaute of the app. We integrated a pre trained tensorflow object detection model to our UI. I learned alot of new things in various domains like ML and small introduction to Android Studio. I would just like to thank SkillUp for providing us with this opportunity and opening so many domains for learning."
- Vriddhi Gupta: - "It was a surreal experience for me. While working on this project I learnt implemenating ML models like facial recognition model. We couldn't use API for facial recognition feature since all were paid , hence I along with my partner Krishna implmented the entire facial recognition program from scratch but we could not add this feature into our app because we still need to implement server client feature in order to use this model with the app. It was a great expereince while working on the UI with my UI team. I learnt the fact that as a part of a team we should always be ready to help each other."
- Krishna Asher: "Working on this project with the CognEye team was an incredible opportunity for me. I finally got my courage up and started working on Ml concepts, learning how to apply machine learning models such as facial recognition, thanks to the encouragement of our mentor Shreyansh and the determination displayed by my fellow team mates to build an incredible project. Vriddhi and I designed our own facial recognition model from the ground up because we couldn't use APIs for facial recognition because they were all paid. However, we were unable to integrate this model into our app because, in order to do so, we needed to create a web server application to handle the requests from our app ."
- Chanchal Agrawal: "I learned alot of new things in the course of this project. Everybody pitching in their new ideas and implementing them was fun. I was part of the OCR team and got to learn about new libraries and APIs. I also learned about front end development and it was overall a great experience."
- Aishwarya Harkare: "Collaborating with team CognEye was wonderful experience for me! In this project meant for people with low vision I , along with my colleague Purva Anjarlekar implemented the Optical Character Recognition(OCR) feature. We used Google Mobile vision API available online for this purpose and integrated it with the app. Also I contributed in making UI for the app. Co-ordinating with the team was a great experience and I was able to dive into whole new world of ML and App development while working on this project."
- Purva Anjarlekar: "I had just started learning ML when I joined this group project. It helped me a lot with implementation and understanding concepts. We specifically didn't work on developing a model but learning to work with APIs and integrating it with the app was very interesting to do. Thank you for this opportunity provided by SkillUp compelling me to study the subject."
- We will deploy the facial recognition model that is already implemented.
- Voice activation for opening the app and using it's feature can be also added to this app making it easy to use for visually impaired people.
- We will also be testing our applcation on certain visual impared people to do more improvisation.
- We will also be working on more features like location detection, Voice based QR code scanner, etc.
.jpeg)
.jpeg)
.jpeg)
.jpeg)
