Model pruning is a common step when deploying a DL DSP system. When a network is trained, a much higher capacity (more trainable parameters) is often needed than what is actually needed at runtime to achieve the same validation metrics (with validation accuracy being the most common). Pruning is essentially setting some weights to 0 reducing the inference computational load, and, depending on the model structure, the dimensions of the trainable sections can often be reduced. There are many approaches to model pruning and it is an important and active area of current research.
This week you will apply at least “magnitude pruning.” You may try additional methods if you wish. Magnitude pruning often works surprisingly well given its simplicity. Given a proportion, it sets that proportion of model weights to 0 by selecting the ones with the smallest magnitude. You will be following this MATLAB example and applying it to any of the DL classifiers from last week’s lab. (If you did not design a DL classifier in last week’s lab, the professor recommends building the one based on AlexNet that classifies the PNG images provided in the .zip file; you should be able to generate a reasonable accuracy (>75%) network fairly quickly.)
Note that you can use the MATLAB
save filebasename command to save all the variables in the workspace to filebasename.mat. You can also save select variables with
save my_trained_network_1 moreVariableNames. Use
load filebasename to load the file the next time you run MATLAB, or to return to a previous state after issuing
There are several short helper functions given near the end of the tutorial. You can either copy and paste these into the MATLAB editor and save them as .m files or download the .zip of them posted on Canvas and unzip it into your MATLAB working directory.
A few adaptations will need to be made to the tutorial to work with the data and network formats from week 2.
preprocessMiniBatch.massumes 4-D data (height, width, channel, sampleNumber). It might be necessary to modify it if using other data formats. You’ll likely get an error if this is necessary.
Submit your informal report covering the above items via Canvas as a Word file, PDF, etc.