Inspiration
Though you've long-since figured out how to physically put your clothes on, matching the colors and the styles of your outfit may be a total mystery. As we ourselves may have spent huge amounts of time fidgeting with the art of clothing, we feel that a fashion app sepecializing in creating a perfect daily look for people could be our solution.
How it works
In particular, we want to extract some features of the good clothes matching and use machine learning to generalize which combinations of the features that yield to good matching. In this case, we extract two features: the category of clothes and the color schemes.
Then, we divide the task of matching clothes into two steps, each deal with one feature.
First, given an image of clothes on a model, the machine needs to classify the category of the clothes. Is the model in jeans or a t-shirt? This is an supervised learning task. We trained an Artificial Neural Network with a large amount of clothes images and their category label.
Second, the AI needs to analyze the color schemes of all photos in our database and decide whether the color matching of a new data is good. This is an unsupervised learning task because all the images in our database are good color matching. We are unable to collect a large amount of bad color matching data. Therefore, we decide to use kmeans to do clustering on the color schemes of the images in our database and compare the color scheme of new data with the existing clusters. If the new color scheme is in the cluster, the AI will evaluate it to be a good color matching; otherwise, the AI will give recommendations of color matching which is the closest clustering.
On the webpage, we used d3 to visualize the way our AI learns. Under the root node, the nine children represent for a category of clothing, and the children of them represent for color schemes. The leaf nodes are images.
We also showed a demo of how our AI recommend clothes. At this stage, for a picture a user upload, we first use AlchemyAPI to check whether that is a picture of clothing. Then, we will use the AI to evaluate the picutre. The AI is able to recognize the category of clothes with a correctness rate of 90% and then evaluate the color by comparing the color scheme with existing clusters and recommend similar images.
Challenges I ran into
Using AI algorithms to teach DeepFashion is definitely the biggest challenge we encounted. Image recognition it self is not a easy task, not mentioning the fact that we want the AI to extract and analyze a relatively large amount of information just from input image files. Extracting features from input data, relatively small training data size, and transformation from training results into user-understandable output are all factors we need to consider. In addition, how to create a user-friendly user interface is another challenge we have. At the very end, we want DeepFashion to be an educational and intelligent AI model. We expect the work flow of taking user input, processing data, and providing proper response to be natural and intuitive.
Accomplishments that I'm proud of
Building something from scratch and making it work is always rewarding. Using AI algorithms has once seemed so distant, so I am very proud of the fact that we have managed to use AI to help people with daily problems using carefully designed data visualization methods. Along the way, we have come across many challenges, but we finally managed to navigate through and buit a beautiful and intelligent web-app. We are proud the we made it.
What I learned
As college students, my teammate and I felt this hackathon to have been an especially great learning experience. This hackathon has been two of my teammates’ first and has been an exploration for all four of us. First, we learned some new languages and skills that are not taught in school, like D3JS and other APIs. It is also exciting to see how school knowledge can be related to building something that might really be useful for people. But, what we learned from this hackathon is far more than techinical skills. Being exposed to the intense hacking environment, we all felt motivated and inspired by other hackers’ passion and felt the excitment of programming.
What's next for DeepFashion
The core of DeepFashion is the AI algorithm behind the scene. At the current stage, it has a good performance on style recognition and color clustering. However, in order to be a real-intelligent outfit counselor, there are more works required. The AI algorithm can definitely be proved and we have to increase the size of the training data set. Once DeepFashion is smart enough, we want to incorporate noisy user inputs and potentially integrate e-commerce or social network into this AI model.
Log in or sign up for Devpost to join the conversation.