Note
Click here to download the full example code
Load and predict with ONNX Runtime and a very simple model¶
This example demonstrates how to load a model and compute the output for an input vector. It also shows how to retrieve the definition of its inputs and outputs.
import onnxruntime as rt
import numpy
from onnxruntime.datasets import get_example
Let’s load a very simple model. The model is available on github onnx…test_sigmoid. Note: The next release (ORT 1.10) will require explicitly setting the providers parameter if you want to use execution providers other than the default CPU provider (as opposed to the current behavior of providers getting set/registered by default based on the build flags) when instantiating InferenceSession. Following code assumes NVIDIA GPU is available, you can specify other execution providers or don't include providers parameter to use default CPU provider.
example1 = get_example("sigmoid.onnx")
sess = rt.InferenceSession(example1, providers=["CUDAExecutionProvider"])
Let’s see the input name and shape.
input_name = sess.get_inputs()[0].name
print("input name", input_name)
input_shape = sess.get_inputs()[0].shape
print("input shape", input_shape)
input_type = sess.get_inputs()[0].type
print("input type", input_type)
Out:
input name x
input shape [3, 4, 5]
input type tensor(float)
Let’s see the output name and shape.
output_name = sess.get_outputs()[0].name
print("output name", output_name)
output_shape = sess.get_outputs()[0].shape
print("output shape", output_shape)
output_type = sess.get_outputs()[0].type
print("output type", output_type)
Out:
output name y
output shape [3, 4, 5]
output type tensor(float)
Let’s compute its outputs (or predictions if it is a machine learned model).
import numpy.random
x = numpy.random.random((3,4,5))
x = x.astype(numpy.float32)
res = sess.run([output_name], {input_name: x})
print(res)
Out:
[array([[[0.56617785, 0.551158 , 0.57431483, 0.62868774, 0.5294609 ],
[0.6545371 , 0.64250827, 0.6819708 , 0.5105157 , 0.5584753 ],
[0.66830933, 0.7094791 , 0.70664704, 0.6744693 , 0.7030401 ],
[0.5395019 , 0.7210481 , 0.5845876 , 0.59664494, 0.6563896 ]],
[[0.71235013, 0.6528918 , 0.5907483 , 0.66855776, 0.61100346],
[0.51468205, 0.60125333, 0.5410304 , 0.57149607, 0.56778824],
[0.5155948 , 0.54921585, 0.5138594 , 0.7051111 , 0.62632954],
[0.5651827 , 0.55247986, 0.6941072 , 0.50415695, 0.7062323 ]],
[[0.51758766, 0.67160237, 0.59442437, 0.5007695 , 0.56175166],
[0.72844744, 0.5150477 , 0.5052765 , 0.5447472 , 0.7088654 ],
[0.596162 , 0.5197903 , 0.6099661 , 0.724396 , 0.5885481 ],
[0.6910895 , 0.53817046, 0.596786 , 0.6119356 , 0.5707261 ]]],
dtype=float32)]
Total running time of the script: ( 0 minutes 0.012 seconds)
