|In the age of information, data abundance has enabled scientists to create models that have great positive impact in our life and society. However, many times the rate of data production is much bigger than the rate of classifying them in the appropriate label due to complexity, personnel or cost of equipment for the labeling task. For this reason, Active Learning techniques have been developed with the Uncertainty Sampling being one of the most popular techniques for querying the unlabeled data. However, selecting the correct Query Strategy for ranking the uncertainty in order to create the best possible model is a time and cost consuming task and most of the times the Active Learning process needs to be repeated multiple times during training. In this work, we exploit the Meta-Features extracted by 123 datasets and select the winning Query Strategy among Least Confidence, Smallest Margin and Entropy for each dataset. In the sequence, we create a dataset with a subset of the extracted Meta-Features and the winning Query Strategy for each dataset and train it in order to create a Decision Tree that can be used in order to select the most suitable Query Strategy.
*** Title, author list and abstract as seen in the Camera-Ready version of the paper that was provided to Conference Committee. Small changes that may have occurred during processing by Springer may not appear in this window.