Training data is the key input to machine learning and having the right quality and quantity of data sets is important to get accurate results. In the planning stages of a machine learning problem, the team is usually excited to talk about algorithms and deployment infrastructure. Much effort is spent discussing the tradeoffs between various approaches and algorithms. Eventually, the project gets off the ground, but then the team often runs into a roadblock. They realize that the data available to train the deep learning models are not sufficient to achieve good model performance. To move forward the team needs to collect more data.
Every machine learning vision method is built around a significant collection of labeled photos, regardless of whether the issue at hand is image classification, object detection, or localization. But when confronting deep learning issues in computer vision, designing a data collecting strategy is a crucial step that is frequently skipped. Don’t be mistaken One of the biggest obstacles to a successful applied deep learning project is assembling a high-quality dataset.
Factors involved to make a good dataset
A good dataset for machine learning projects has three keys: quality, quantity, and variability.
Quality
Quality images will replicate the lighting, angles, and camera distances that would be found in the target location. A high-quality dataset contains distinct examples of the desired topic. Generally speaking, if you are unable to recognize your target subject from an image, neither can an algorithm. This rule has some major exceptions, such as recent developments in face recognition, but it’s an excellent place to start.
If the target object is tough to see, consider adjusting the lighting or camera angle. You may also consider adding a camera with optical zoom to enable closer images with greater detail of the subject. In the image shown below, we can see low resolution vs high-resolution images. If you train the model on poor quality low resolution images, it would make the model difficult to learn. Whereas good quality images help the model to easily get trained on the classes we wish for. The efficiency and time required to train the model are affected by the quality of the dataset being used.
Quantity
Each parameter that your model has to consider in order to perform its task increases the amount of data that it will need for training. Generally, the more labeled instances available for training vision models the better. Instances refer to not just the number of images, but the examples of a subject contained in each image. Sometimes an image may contain only one instance as is typical in classification problems such as problems classifying images of cats and dogs.
In other cases, there may be multiple instances of a subject in each image. For an object detection algorithm, having a handful of images with multiple instances is much better than having the same number of images with just one instance in each image. As a result, the training method you use will cause significant variation in the amount of training data that is useful to your model.
Variability
The more variety a dataset has, the more value that dataset can provide to the algorithm. A deep learning vision model needs variety in order to generalize to new examples and scenarios in production. Failure to collect a dataset with variety can lead to overfitting and poor performance when the model encounters new scenarios. For example, a model that is trained based on daytime lighting conditions may show good performance on images captured in the day but will struggle under nighttime conditions. In the example below we have shown how various timing and light conditions gives us varied image dataset and we are able to train the model to give accurate predictions in all varied conditions.
Models may also be biased if one group or class is overrepresented in the dataset. So whenever the model encounters a different scenario in which it is not trained, the prediction is failed. This is common in face detection models where most facial-recognition algorithms show inconsistent performance across subjects that vary by age, gender, and race Having a dataset with good variety not only leads to good performance but also helps address potential issues related to consistent performance across the full range of subjects.
How to build a good dataset?
The process of creating a dataset involves three important steps:
1: Data Collection
2: Data Cleaning
3: Data Labeling
1: Data Collection
The process of data acquisition involves finding datasets that can be used for training machine learning models. There are a couple of ways you can go about doing this, and your approach will largely depend on the problem that you are trying to solve and the type of data that you think is best suited for it.
Don’t underestimate the difficulty of collecting a high-quality dataset. Collecting enough examples can be time-consuming and expensive. Even with a good data collection process, it could take weeks or months to collect enough instances to achieve good model performance across all representative classes. This is particularly true when you are trying to capture examples of rare events, such as examples of bad quality in a manufacturing line.
2: Data cleaning
If you do have enough data, but the quality of the dataset isn’t that great (e.g., data is noisy), or there’s an issue with the general formatting in your dataset (e.g., some data intervals are in minutes while some in hours), we move on to the second most important process, which involves cleaning the data.
You can perform data operations manually, but it is labor-intensive and would take a lot of time. Alternatively, you can leverage already built systems and frameworks to help you achieve the same goal easier and faster. Since missing values can tangibly reduce prediction accuracy, make this issue a priority.
3: Data labeling
Data Labeling is an important part of data preprocessing that involves attaching meaning to digital data. Input and output data are labeled for classification purposes, and provide a learning basis for future data processing. For example, the picture of a dog can be attached to the label “a dog”.Now you have acquired enough data to have a representative dataset (a dataset that captures the most important information), clean, and in the right format.
Depending on the task you’re doing, data points can be annotated in different ways. This can cause significant variation in the number of labels your data produces, as well as the effort it takes to create those labels. TagX creates digital data assets powering Artificial Intelligence by collecting, annotating, analyzing, and pre-processing data corpus for training, evaluation, and test purposes.
Build Quality Assurance into your labeling process
Many applications for deep learning in vision require labels that identify objects or classes within the training images. Labeling takes time and requires consistency and careful attention to detail. Poor quality in the labeling process could be due to several causes, all of which can lead to poor model performance. Untagged instances and Inconsistent bounding boxes or labels are two examples of poor labeling quality.
To help ensure labeling quality, build a “review” step into the labeling process. Has each label been reviewed by at least one other person than the labeler to help protect against bad quality in the labeling process?
Increased use of Synthetic Data
Great progress has been made in recent years in simulating realistic images. Simulators have been used to help train models for self-driving cars and robotics problems. These simulations have become so good that the resulting images can be used to support training deep learning models for computer vision. These images can augment your dataset and, in some cases, even replace your training dataset.
This is an especially powerful technique for deep reinforcement learning, where the model must learn a wide variety of training examples. Synthetic data is Fusing computer graphics and data generation technologies to simulate real-world scenarios with photo-realistic details. TagX generates these datasets to propel Machine Learning Algorithms faster to production.
Final Thoughts
Once you have a large-high-quality dataset you can focus on model training, tuning, and deployment. At this point, the hard effort of collecting and labeling images can be translated into a working model that can help solve your computer vision problem. After spending days or even weeks collecting images, the training process will go fast by comparison. Continue to evaluate your models as you collect more images to maintain a sense of progress. This will give you an idea of how your model is improving and allow you to gauge the value of more training images.
TagX offers complete Data Solutions right from collection to labeling to tweaking datasets for better performance, book a consultation call today to know more.
No comments yet