![]() The optimisation process removes layers in the model that were only required during training and can also recognise when a group of layers could be unified into a single layer, which decreases the inference time. OpenVINO supports trained models from a wide range of common software frameworks like TensorFlow, MXNet and Caffe. In addition to the toolkit, there is the Open Model Zoo repository on GitHub, which comprises a set of pre-trained deep learning models that organisations can use freely, plus a set of demos to help with development of new ones. The Model Optimizer converts a trained neural network from a source framework to an open-source, nGraph-compatible intermediate representation ready for use in inference operations. The two main components of OpenVINO are its Model Optimizer and the Inference Engine. “So, the shift we did last year was to make OpenVINO more accessible and for developers to be able to focus solely on the problem that they’re trying to solve and not be encumbered by having to pick different toolkits for different workloads,” Cayetano says. “So they needed a toolkit that would have everything that you want from a computer vision library that allows you to resize that image, or that allows you to quantize your model into a different data type like INT8 for better performance.”Īnd although OpenVINO started off with a focus on computer vision, the feedback Intel got from developers was that they wanted a more general-purpose solution for AI, whether that was for applications involving image processing, audio processing or speech and even recommendation systems. But there was a gap in developer efficiency where developers had to use a variety of different tools and ensure they were interoperable and could operate with each other in the same way,” she says. Interoperability is another issue, according to Cayetano, “We were seeing this trend of a lot of businesses adopting AI and wanting to take AI into production. The camera may be positioned further away so the images may have to be adjusted – which OpenVINO can do. As an example, Cayetano cites a defect detection scenario in which a camera view close to the production line may have been assumed during training. In addition, the model may have been trained in ideal circumstances that differ from the actual deployment environment. ![]() This is because the trained models are shoe-horned into a deployment without considering the system they are running on. There’s a variety of different niche challenges in inferencing that we’ve tackled with OpenVINO, that are different from when models and applications are in the training phase,” she says.įor example, Intel has found there is often a sharp decline in accuracy and performance when models that were trained in the cloud or in a datacentre are deployed into a production environment, especially in an edge scenario. “That’s really useful when you’re taking an AI application into production. It can also fine tune the model for the platform the customer wants to use, claims Zoë Cayetano, Product Manager for Artificial Intelligence & Deep Learning at Intel. This means that it acts like an abstraction layer between the application code and the hardware. OpenVINO takes a trained model, and optimises it to operate on a variety of Intel hardware, including CPUs, GPUs, Intel® Movidius™ Vision Processing Unit (VPU), FPGAs, or the Intel® Gaussian & Neural Accelerator (Intel® GNA). One solution to these problems is to employ OpenVINO™ (Open Visual Inference & Neural Network Optimization), a toolkit developed by Intel to speed the development of applications involving high-performance computer vision and deep-learning inferencing, among other use cases. Especially for situations such as edge deployments, where less compute power is available than in a datacentre. Another complication is how to deploy a model onto a different system than the one that was used to train it. From chatbots forming the first line of engagement in customer services, to image recognition systems that can identify defects in products before they reach the end of the production line in a factory.īut many organisations are still stuck at where to start in building machine-learning and deep-learning models and taking them all the way from development through to deployment. Sponsored Artificial Intelligence techniques have been finding their way into business applications for some time now.
0 Comments
Leave a Reply. |