Click here to Skip to main content
15,124,972 members
Articles / Programming Languages / C#
Technical Blog
Posted 26 Nov 2017

Tagged as


2 bookmarked

How to Save CNTK Model to File in C#

Rate me:
Please Sign up or sign in to vote.
5.00/5 (4 votes)
26 Nov 2017CPOL3 min read
How to save CNTK model to file in C#

The final process of training is the model which should be used in the production as safe, reliable and accurate. Usually, model training is a frustrating and time consuming process, and it is not like we can see as demo to introduce with the library. Once the model is built with the right combination of parameter values and network architecture, the process of modeling turns into a interesting and funny task, since it calculates the values just as we expect.

In most of the time, we have to save the current state of the model, and continue with training due to various reasons:

  • to change the parameters of the learner
  • to switch from one to another machine
  • or to share the state of the model with your team mate
  • to switch from CPU to GPU
  • etc.

In all the above cases, the current state of the model should be saved, and continue the training process from the last stage of the model. Because, the training from the beginning is not a solution, since we already invest time and knowledge to achieve progress of the model building.

The CNTK supports two kinds of persisting the model.

  • production/ready or evaluation ready state, and
  • saving the checkpoint of the model for later training

In the first case, the model is prepared for the evaluation and production but cannot be trained again, because it is freed from all other information but for the evaluation. During the saving process, only one file is generated.

In the second case, beside a model file, another file is generated with the name “modelname.ckp”. The file contains all information needed for the continuation of training. Once the trainer checkpoint is persisted, we can continue with model training even if we changed the following:

  • the training data set with the same dimensions and data types
  • the parameters of the learner
  • the learner

What we cannot change in order to continue with training is the network model. In other words, the model must remain with the same number of layers, input and output dimensions.

Saving, Loading and Evaluating the Model

Once the model is trained, it can be persisted as separated file. As separate file, it can be loaded and evaluated with different dataset, but the number of the features and the label must remain the same as in case when was trained. Use this method when you want to share the model with someone else, or when you want to deploy the model in the production.

The model is saved simply by calling the CNTK method Save:

public void SaveTrainedModel(Function model, string fileName)

The model evaluation requires several steps:

  • load the model from the file
  • extract the features and label from the model
  • call evaluate method from the model, by passing the batch created from the features, label and the evaluation dataset

The model is loaded by calling Load method.

public Function LoadTrainedModel(string fileName, DeviceDescriptor device)
   return Function.Load(fileName, device, ModelFormat.CNTKv2);

Once the model is loaded, features and label are extracted from the model in the following way:

//load the model from file
Function model = Function.Load(modelFile, device);
//extract features and label from the model
Variable feature = ffnn_model.Arguments[0];
Variable label = ffnn_model.Output;

The next step is creating the minibatch in order to pass the data to the evaluation. In this case, we are going to create only one row for the Iris example of:

//Example: 5.0f, 3.5f, 1.3f, 0.3f, setosa
float[] xVal = new float[4] { 5.0f, 3.5f, 1.3f, 0.3f };
Value xValues = Value.CreateBatch<float>(new int[] {feature.Shape[0] }, xVal, device);
//Value yValues = - we don't need it, because we are going to calculate it

Once we created the variable and values, we can map them and pass to the model evaluation, and calculate the result:

//map the variables and values
var inputDataMap = new Dictionary<Variable, Value>();
var outputDataMap = new Dictionary<Variable, Value>();
outputDataMap.Add(label, null);
//evaluate the model
ffnn_model.Evaluate(inputDataMap, outputDataMap, device);
//extract the result  as one hot vector
var outputData = outputDataMap[label].GetDenseData<float>(label);

The evaluation result should be transformed to proper format, and compared with the expected result:

//transforms into class value
var actualLabels = outputData.Select(l => l.IndexOf(l.Max())).ToList();
var flower = actualLabels.FirstOrDefault();
var strFlower = flower == 0 ? "setosa" : flower == 1 ? "versicolor" : "virginica";
Console.WriteLine($"Model Prediction: 
                  Input({xVal[0]},{xVal[1]},{xVal[2]},{xVal[3]}), Iris Flower={strFlower}");
Console.WriteLine($"Model Expectation: 
                  Input({xVal[0]},{xVal[1]},{xVal[2]},{xVal[3]}), Iris Flower= setosa");

Training Previous Saved Model

Training previously saved model is very simple, since it requires no special coding. Right after the trainer is created with all necessary stuff (network, learning rate, momentum and other), you just need to call:


No additional code should be added.
The above method is called after you successfully saved the model state by calling:


The method is usually called at the end of the training process.
The complete source code from this blog post can be found here.

Filed under: .NET, C#, CNTK, CodeProject
Tagged: .NET, C#, CNTK, Code Project, CodeProject, Deep Neural Network, Machine Learning


This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


About the Author

Bahrudin Hrnjica
Software Developer (Senior)
Bosnia and Herzegovina Bosnia and Herzegovina
Bahrudin Hrnjica holds a Ph.D. degree in Technical Science/Engineering from University in Bihać.
Besides teaching at University, he is in the software industry for more than two decades, focusing on development technologies e.g. .NET, Visual Studio, Desktop/Web/Cloud solutions.

He works on the development and application of different ML algorithms. In the development of ML-oriented solutions and modeling, he has more than 10 years of experience. His field of interest is also the development of predictive models with the ML.NET and Keras, but also actively develop two ML-based .NET open source projects: GPdotNET-genetic programming tool and ANNdotNET - deep learning tool on .NET platform. He works in multidisciplinary teams with the mission of optimizing and selecting the ML algorithms to build ML models.

He is the author of several books, and many online articles, writes a blog at, regularly holds lectures at local and regional conferences, User groups and Code Camp gatherings, and is also the founder of the Bihac Developer Meetup Group. Microsoft recognizes his work and awarded him with the prestigious Microsoft MVP title for the first time in 2011, which he still holds today.

Comments and Discussions

QuestionOdd error loading model that incorporates SoftMax Pin
asiwel30-Nov-17 17:31
professionalasiwel30-Nov-17 17:31 
AnswerRe: Odd error loading model that incorporates SoftMax Pin
Bahrudin Hrnjica30-Nov-17 21:26
professionalBahrudin Hrnjica30-Nov-17 21:26 
GeneralRe: Odd error loading model that incorporates SoftMax Pin
asiwel1-Dec-17 8:21
professionalasiwel1-Dec-17 8:21 
GeneralRe: Odd error loading model that incorporates SoftMax Pin
Bahrudin Hrnjica2-Dec-17 5:14
professionalBahrudin Hrnjica2-Dec-17 5:14 
I think SaveCheckpoint method generates two files. One is the model and second id the .chk file.

Since you are using the same file name for Save() and SaveCheckpoint your model for production is rewritten by the model which is produced with the SaveCheckpoint method.
GeneralRe: Odd error .. Well, well, surprise! Pin
asiwel2-Dec-17 7:49
professionalasiwel2-Dec-17 7:49 
QuestionHow to obtain output probability estimates? Pin
asiwel27-Nov-17 14:06
professionalasiwel27-Nov-17 14:06 
AnswerRe: How to obtain output probability estimates? Pin
Bahrudin Hrnjica28-Nov-17 2:00
professionalBahrudin Hrnjica28-Nov-17 2:00 
GeneralRe: How to obtain output probability estimates? Pin
asiwel28-Nov-17 9:03
professionalasiwel28-Nov-17 9:03 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.