ML.NET and Machine Learning
In this post we are going to describe about Machine learning and why it is used? Then overview of ML.NET and how it is used for Machine learning.
Overview of Machine Learning
Machine learning is getting computers to make predictions without being explicitly programmed. Machine learning is used to solve problems that are difficult (or impossible) to solve with rules-based programming (e.g., if statements and for loops). For instance, if you were asked to create an application that predicts whether an image has a dog in it or not, you might not know where to start. Similarly, if you were asked to write a function that predicts the price of a shirt based on the description of the shirt, you might start by looking at keywords such as “long sleeves” and “business casual,” but you might not know how to build a function to scale that to a few hundred products.
Even if you don’t know how to write that function, you have examples of inputs and outputs to the function that you can use to create a model. Machine learning simply means taking in historical data and using algorithms to find patterns and rules in the data (e.g., training a model) that can then be applied to new data to make predictions (e.g., consuming a model).
With machine learning, you can automate a large range of “human” tasks, like classifying images of dogs and other objects or predicting values such as the price of a car or house, which allows you to analyze a large amount of data more quickly and accurately than ever before and to more easily make data-driven decisions.
Overview of ML.NET
ML.NET is first and foremost a framework that you can use to create your own custom ML models. This custom approach contrasts with “pre-built AI,” where you use pre-designed general AI services from the cloud (like many of the offerings from Azure Cognitive Services). This can work great for many scenarios, but it might not always fit your specific business needs due to the nature of the machine learning problem or to the deployment context (cloud vs. on-premises).
With ML.NET, you have the ability to create and train your own ML models, which makes ML.NET very flexible and adaptable to your specific data and business domain scenarios. In addition, because the framework comprises libraries and NuGet packages to use in your .NET applications, you can run ML.NET anywhere.
Made for .NET Developers
ML.NET enables developers to use their existing .NET skills to easily integrate machine learning into almost any .NET application. This means that if C# (or F# or VB) is your programming language of choice, you no longer have to learn a new programming language, like Python or R, in order to develop your own ML models and infuse custom machine learning into your .NET apps. The framework offers tooling and features to help you easily build, train, and deploy high-quality custom machine learning models locally on your computer without requiring prior machine learning experience.
Run It Anywhere for Free
Because ML.NET is a free and open-source framework (comparable in autonomy and context to other frameworks in .NET like Entity Framework, ASP.NET, or even .NET Core), you can use it wherever you want and in any .NET application. This includes Web apps and services (ASP.NET MVC, Razor Pages, Blazor, Web API), desktop apps (WPF, WinForms), and more.
This means that you can build and consume ML.NET models on-premises or on any cloud, such as Microsoft Azure. In addition, because ML.NET is cross-platform, you can run the framework on any OS environment: Windows, Linux, or macOS. You can even run ML.NET on the traditional .NET Framework on Windows in order to infuse AI/ML into your existing .NET applications.
This also applies to offline scenarios; you can train and consume ML.NET models in offline scenarios such as desktop applications (WPF and WinForms) or any other offline .NET app (excluding ARM processors, which are currently not supported).
Interoperability with Python and Data Scientists
ML.NET also offers Python bindings called NimbusML. If your organization has teams of data scientists who are more skilled in Python, they can create ML.NET models in Python with NimbusML, which you can then consume in your production end-user .NET applications very easily while running it as native .NET.
ML.NET Components
ML.NET fundamentally provides a .NET API that you can use for two main types of actions:
- ML model training: Creating/building the model, usually in your “back storage”
- ML model consumption: Using the model to make predictions in end-user applications in production. following figure shows the ML.NET major components:
As seen in the figure above, ML.NET has several major components.
- Data components:
IDataView: Data in ML.NET is represented as an IDataView, a flexible, efficient way of describing tabular data (e.g., rows and columns). The IDataView provides a placeholder to load datasets for further processing. It’s designed to efficiently handle high-dimensional data and large data sets and is therefore the component that holds the data during data transformations and model training. Although you can load data from a file or enumerable into an IDataView, you can also stream data from the original data source while training without needing to load all the data in-memory, so it can handle huge data sets up to terabytes in size. IDataView objects can contain numbers, text, Booleans, vectors, and more.
Data Loaders: You can load datasets into an IDataView from virtually any data source. You can use File Loaders for typical sources in ML, like text, binary, and image files, or you can use the Database Loader to load and train data directly from any relational database supported by System.Data, such as SQL Server, Oracle, PostgreSQL, MySQL, etc.
Data Transforms: Because machine learning is all about math, all data needs to be converted to numbers or numeric vectors. ML.NET provides a variety of data transforms, such as text featurizers and one hot encoders, to convert your data to an acceptable format for the ML algorithms.
2. Training ML tasks components:
Classical ML tasks: ML.NET supports many classical machine learning scenarios and tasks, such as classification, regression, time series, and more. ML.NET provides more than 40 trainers (algorithms targeting a specific task), so you can select and fine-tune the specific algorithm that achieves higher accuracy and better solves your ML problem.
Computer Vision: Starting in ML.NET 1.4-Preview, ML.NET also offers image-based training tasks (image classification/recognition) with your own custom images, which uses TensorFlow under the covers for training. Microsoft is working on adding support for object detection training as well.
3. Model Consumption and Evaluation components:
Model consumption: Once you’ve trained your custom ML model, ML.NET provides several ways to make predictions, such as using the model itself to make predictions in bulk, using the Prediction Engine to make single predictions, or using the Prediction Engine Pool for making predictions in scalable and multi-threaded applications.
Model evaluation: Before using a trained model in production, you want to make sure it achieves the required quality when making predictions. ML.NET provides multiple evaluators related to each ML task so that you can find out the accuracy of your model, plus many more typical machine learning metrics depending on the targeted ML task.
4. Extensions and Tools:
ONNX model integration: ONNX is a standard and interoperable ML model format. You can run/score any pre-trained ONNX model in ML.NET.
TensorFlow model integration: TensorFlow is one of the most popular deep learning libraries. In addition to the image classification training scenario previously mentioned, you can also run/score any pre-trained TensorFlow model with this API.
Tools: You can use ML.NET’s tools (Model Builder in Visual Studio or the cross platform CLI) to make model training even easier. These tools use the ML.NET AutoML API internally to try many different combinations of algorithms and configurations in order to find the best model for your data and scenario.
Supported ML Tasks and Scenarios
You can use ML.NET to build custom models for many scenarios grouped by various ML tasks.
Conclusion
In this post you’ve seen an overview of Machine learning and ML.NET and the different components of the framework.
In my next post I am going to explain Programing steps in ML.NET ,which can describe the different steps, create, train, evaluates, and consumes the ML model.
This post is part of “ML.NET-Step by step”.