Neural networks show promise in modelling infrared (IR) spectroscopic data, but many of the proposed solutions in the literature are either Multilayer Perceptrons with a single hidden layer or are adapted from successful solutions in other domains such as convolutional networks from image processing. This thesis investigates the use of Monte Carlo search to build candidate deep neural networks from scratch. As the prediction targets are real numbers and the modelling is prone to error from outliers, lazy, locally weighted methods are also investigated. Predictions from the network are compared to predictions made from locally weighting the features extracted from the same network. To measure progress, these results are compared to classical statistical methods commonly used to model spectroscopic data, such as partial least squares (PLS) regression. Results suggest that deep networks trained via Monte-Carlo search outperform classical approaches. They also show that PLS dimension reduction as a preprocessing technique can improve the quality of search at little cost in predictive ability. Encouraging results using this approach are obtained on the publicly available Mangoes dataset. Finally, a streaming regression solution is investigated in which selected deep neural network models perform well. In this setting, however, locally weighted regression can result in sporadic outlier predictions, so an ensemble solution is proposed as a remedy.